KeepCurrent

We organize #meetups, #hackathons and conduct #tailormade #trainings and #workshops for companies: #MachineLearning #MLOps, #DataScience & #SoftwareDevelopment.

[Hila Gonen @hila_gonen]

Do you like yellow? Then, according to LLMs, you are probably a school bus driver!
Excited to share our new paper about Semantic Leakage in Language Models!
Joint work with my wonderful collaborators @terra @alisawuffles @luke @nlpnoah

Paper: https://t.co/Y0ytAZMcEu

1/10

[Vaibhav (VB) Srivastav @reach_vb]

Probably the craziest week in Open Source AI (yet):

1. Mistral (in collaboration with Nvidia) dropped Apache 2.0 licensed NeMo 12B LLM, better than L3 8B and Gemma 2 9B. Models are multilingual with 128K context and a highly efficient tokenizer - tekken.

2. Apple released DCLM 7B - truly open source LLM, based on OpenELM, trained on 2.5T tokens with 63.72 MMLU (better than Mistral 7B)

3. HF shared SmolLM - 135M, 360M, & 1.7B Smol LMs capable of running directly in the browser; they beat Qwen 1.5B, Phi 1.5B and more. Trained on just 650B tokens.

4. Groq put out Llama 3 8B & 70B tool use & function calling model checkpoints - achieves 90.76% accuracy on Berkely Function Calling Leaderboard (BFCL). Excels at API usage & structured data manipulation!

5. Salesforce released xLAM 1.35B & 7B Large Action Models along with 60K instruction fine-tuning dataset. The 7B model scores 88.24% on BFCL & 2B 78.94%

6. Deepseek changed the game with v2 chat 0628 - The best open LLM on LYMSYS arena right now - 236B parameter model with 21B active parameters. It also excels at coding (rank #3) and arena hard problems (rank #3)

There's a lot more; Arcee (mergekit) released a series of LLMs, each better than the other, and Numina and HF Numina 72B (based on Qwen 2) and Math datasets, Mixbread with embedding models (english + german) and a lot more!

It's fun to see so many releases next week with L3 405B (?) and companions; we might see a shift in the Open LLM landscape! See you next week!

What else did I miss? 🤗

[Tiago Pimentel @tpimentelms]

Are you interested in word lengths and natural language’s efficiency? If yes, our new #EMNLP2023 paper has everything you need: drama, suspense, a new derivation of Zipf’s law, an update to Piantadosi et al’s classic word length paper, transformers... 🧵

https://t.co/11wxFuBX6T

[Zhiqiu (Oscar) Xu @oscar_zhiqiu_xu]

You don’t have to train from scratch whenever developing a smaller model of an existing model family.

Sharing our latest work - “Initializing Models with Larger Ones”

arxiv preprint: https://t.co/lSv1WK4Dl3
code: https://t.co/GDDRHJ5C4U

[Aviv Slobodkin @NeurIPS @lovodkin93]

🎉Excited to announce our paper's acceptance at #EMNLP2023! We explore a fascinating question: When faced with (un)answerable queries, do LLMs actually grasp the concept of (un)answerability?🧐 This work is a collaborative effort with @clu_avi @ravfogel @omerNLP and Ido Dagan 1/n

[Samuel Müller @SamuelMullr]

There is a paper by Google trending right now, that claims transformer in-context learning cannot generalize between two function classes

I have reproduced their experiment in a colab and come to a very different conclusion...

[ACL 2026 @aclmeeting]

ACL org announcement: 📢The list of accepted workshops in the ACL Conferences (@aclmeeting, @eaclmeeting, @naaclmeeting, @emnlpmeeting) in 2024 is out! Please help spread the word. Retweeting w/ references, esp. w/organisers information is very much appreciated - thanks! #NLProc

[Bojan Tunguz @tunguz]

Pandas 2.0 is here! This is the biggest overhaul of Pandas since its inception, and it has been years in the making. However, you will probably not notice too many changes, and all your existing Pandas code will most likely run the same as before. All the major changes are under the hood. That's because Pandas has moved away from the way it represents data, from numpy to Apache Arrow. Pandas was originally built on top of numpy, and it was an adequate solution for many tasks. However, three are many limitations of numpy that have only become more obvious over the years. Apache Arrow will significantly help with those pain points, and will speed up many Pandas tasks.

I've only played with the new version for a day so far. My limited impression is that it significantly speeds up loading and saving of csv files, and puts the new version of Pandas on par with Polars in that regard. Lookign forward to playing more with it in the weeks and months ahead.

Great blog post about what's new in Pandas: https://t.co/9bfJXdDlIl
Release notes: https://t.co/u1JFYSWDHR
GitHub repo: https://t.co/VDGiToKHXc

#DataScience #MachineLearning #Data #Python

[Steve Stewart-Williams @SteveStuWill]

Psychologists have posited hundreds of cognitive biases over the years. A new paper argues that they all boil down to one of a handful of fundamental beliefs coupled with confirmation bias. https://t.co/bDDmNftq7M

[Guillaume Lample @ NeurIPS 2024 @GuillaumeLample]

Today we release LLaMA, 4 foundation models ranging from 7B to 65B parameters.
LLaMA-13B outperforms OPT and GPT-3 175B on most benchmarks. LLaMA-65B is competitive with Chinchilla 70B and PaLM 540B.
The weights for all models are open and available at https://t.co/q51f2oPZlE
1/n

[DeepAI @DeepAI]

Read about Non-Parametric Model https://t.co/ptZkZPYy1h #OrdinalNumber #NonParametricModel

[NeurIPS Conference @NeurIPSConf]

You can now watch the recorded material from #NeurIPS2022 online without registration at:

https://t.co/BtaKedJGbb

[Science Is Strategic @scienceisstrat1]

1/ The 2022 climate data is out. And it’s terrifying.

Overall, 2022 was the fifth warmest year on record.

Below is a short🧵on 2022’s record-breaking droughts, floods, ‘rain bombs,’ wildfires & more

Cc: @dwallacewells @JesseJenkins @ylecun @leahstokes @Noahpinion @MichaelEMann

[Sasha Rush @srush_nlp]

If you are looking for a winter break project, here is the full collection of ML/coding puzzles.
(I think this is more useful than prompting, but who knows?)
* https://t.co/STRN1yPa6G
* https://t.co/1Uogwm4t1m
* https://t.co/MRnLTvEfMA
* https://t.co/Nb6G52dSUa

[Simone Scardapane @s_scardapane]

*Thinking Like Transformers*

Awesome blog post by @srush_nlp based on the paper by the same name.

If you write a programming language inspired by the way Transformers work, how easy would it be to program in it? 👀

Blog: https://t.co/AXEpZQTOHK
Paper: https://t.co/6xyDiFRKU0

[Fangyu Liu @hardy_qr]

📍🧵🚨 QA on plots & charts is a complex task requiring sophisticated reasoning - our visual language models struggle with this.

LLMs are super strong reasoners - but they only work for text.

What do we do? We translate plots & charts to text so LLM can understand!

[Abhilasha Ravichander @lasha_nlp]

🚨Help NLP models process negation🚨

Introducing ℂ𝕆ℕ𝔻𝔸ℚ𝔸, a *contrastive* reading comprehension dataset that requires reasoning about negation

w/ @nlpmattg & @anmarasovic @ai2_allennlp, at #EMNLP2022

📝Paper https://t.co/UfFv1Yi6w0
🚀Data https://t.co/upJUpTNmXY [1/8]

[Aman Jha @amanjha__]

It's here!

Upload *any paper* to Explainpaper and start instantly getting explanations! Ask follow up questions if you need a more in-depth answer.

Go to https://t.co/C7rp4Vj9RS and go read all the papers you've been saving! 📝📝📝

[Mathias Lechner @mlech26l]

We (joint work with @ramin_m_h) have released PyHopper, a hyperparameter tuning platform for streamlining machine learning research.
Pyhopper's goal is to enable people of any skill level to set up advanced multi-GPU hyperparameter tuning processes in less than a minute.

[GenBench @GenBench]

Ever wanted to know more about generalisation in NLP but overwhelmed with the number of papers on ArXiv? Fear not! We read 400+ papers, 600+ experiments, and designed a taxonomy 📝 to categorise the research for you! (1/n) 🧵

https://t.co/qvftIztAWq