Pingjie Tang

[Pingjie Tang @tetetang]

@Nyny16ygx I hope you are using toilet paper to wipe your mouth, cuz it is an ass. Idiot!

[Lance Martin @RLanceMartin]

Great to see folks at the @ollama meetup last night! Gave a lightning talk on a theme we've seen: oss / local LLMs for narrow tasks w/in the RAG stack.

(1) Query transformations
Local / oss LLM can be useful for tasks like query re-writing or decomposition that require reasoning abt a query. Esp interesting for small models (phi-2, etc). See template for one example of query re-writing:
https://t.co/8LMwVjtZcd

(2) Routing
@atroyn mentioned in a talk that routing w/ local / oss LLMs likely to get wrapped in w/ Chroma to route btwn sqlite vs chroma (jointly query on relational and semantic data). Makes a lot of sense.
https://t.co/QOavgK5gdR

(3) Query construction
Text-to-X (SQL, Cypher, Metadata) make a lot of sense for local / oss, esp in cases where the DB is private. Template for text-to-SQL example:
https://t.co/zYMmwV0HwK

(4) Indexing
Tasks related to doc summarization or captioning in the indexing process really good for local / oss to avoid high cost in indexing large corpus. Esp, I like potential for this in multi-modal. Newer LLaVA models (c/o @imhaotian) supporting better OCR (IIRC a recent talk mentioned this is coming) would be great here.
Template:
https://t.co/hFfOkwwTY7
Talk:
https://t.co/02g4eSxYpQ

(5) Post processing
@jerryjliu0 had a nice talk on using local oss LLMs (Mistral) for post-processing (RankGPT). Cool idea, another area that makes a lot of sense.
https://t.co/FOMeKIALnA

Across all of these steps in the flow, @ollama JSON mode can be useful for output parsing. @Hacubu has done some nice working benchmarking this for an eval dataset of email spam. Promising results (e.g., Mixtral 8x-7b can beat GPT3.5 w/ fxn calling).
https://t.co/yRXvlWGtAO

Slides:
https://t.co/RatpCFSYZt

[Pingjie Tang @tetetang]

@_jasonwei Well said, indeed!

[Jason Wei @_jasonwei]

Turns out the ancient Chinese knew a lot about modern neural networks

[Ellen Liu @ellenellen0126]

Take a look at the pandas in China and Japan😕
They were both celebrating their birthdays in the pictures. 38 & 28
Yaya is 22 this year.
Don’t tell me Yaya looks healthy and normal😡
Please help save Yaya!!!!!!
#yayalele #AnimalLovers #Pandas #memphiszoo @PandaVoicesOrg @IDAUSA

[Margorie Burroughs @MargorieBurrou2]

I hope everyone reading this goes to see how the pandas at the Memphis Zoo are being abused compared to the pandas at other zoos, let’s take action to bring yaya home soon! Memphis Zoo starves lele to dead by not giving pandas fresh food, which is outrageous!#Memphis zoo

[Pingjie Tang @tetetang]

@Bobo24682 Korea, you can’t keep stealing because you got nothing…Thanks for becoming a world-class joke for celebrating Chinese New Year！

[Andrej Karpathy @karpathy]

🔥 New (1h56m) video lecture: "Let's build GPT: from scratch, in code, spelled out."
https://t.co/2pKsvgi3dE
We build and train a Transformer following the "Attention Is All You Need" paper in the language modeling setting and end up with the core of nanoGPT.

[Pingjie Tang @tetetang]

There is only ONE China in the world!🇨🇳

[Pingjie Tang @tetetang]

@OhmNanon_CNFC So sad…

[OhmNanon_CNFC @OhmNanon_CNFC]

We are here to bless our 132 compatriots who suffered from MU5735 air crash. We bless them could be in safety. We sincerely bless them could make a magic.We bless all of them could be back safe and sound.🙏🏻🙏🏻🙏🏻
#MU5735

[Jure Leskovec @jure]

Sharing slides from my @textgraphs @NAACLHLT workshop keynote: Reasoning with Language and Knowledge Graphs
https://t.co/GFlRjfZgts

[Lilian Weng @lilianweng]

Contrastive learning aims to learn representation such that similar samples stay close, while dissimilar ones are far apart. It can be applied to supervised / unsupervised data and has been shown to achieve good results on various tasks.

📚 A long read: https://t.co/TZiaVA0qNS

[Sebastian Ruder @seb_ruder]

10 Things You Need to Know About BERT and the Transformer Architecture That Are Reshaping the AI Landscape

This super comprehensive post covers most things that are important in current NLP including BERT, transfer and avocado chairs 🥑
by @cathalhoran

https://t.co/Hn458pkrk8

[Sebastian Ruder @seb_ruder]

ML and NLP Research Highlights of 2020

It's been inspiring to look back on all the exciting advances that happened despite such a tumultuous year. Here's a selection of my highlights.

https://t.co/yxzZCyap3v

[Jeff Dean @JeffDean]

On behalf of the entire Google Research & @GoogleAI communities, I'm excited to share an overview of some of our research in 2020.

Thanks to everyone who helped make this work possible!

https://t.co/EYVeQ7PfO4

[Jiaxuan You @youjiaxuan]

We have just open sourced the code for "Graph Structure of Neural Networks"! It should be pretty clean and easy to use :)
Code: https://t.co/wDatoIK0r7
Paper: https://t.co/chlJX6Qlr3
Slides: https://t.co/5tUwvewT8D

with Jure Leskovec, Kaiming He, Saining Xie, ICML 2020

[Jiaxuan You @youjiaxuan]

We are excited to release #GraphGym, a platform for designing and evaluating #GraphNeuralNetworks.
It provides a modularized pipeline, a system for launching thousands of experiments, and more!

Code: https://t.co/leLZ34vuNO
Paper: https://t.co/lsvOnPxZB1
#NeurIPS2020 Spotlight

[Yi Tay @YiTayML]

As a companion to our recent efficient Transformer survey, we designed "Long Range Arena" a new challenging benchmark to help understand and analyze trade-offs between recent efficient Transformer models. Check out our paper at https://t.co/hnMkari1Oa. @GoogleAI @DeepMind

[Pingjie Tang @tetetang]

The Future of Natural Language Processing https://t.co/9R3IkWRCBn via @YouTube