Philéas Condemine

Data scientist & Actuary : Health + P&C
#Rstats #pytorch #XGboost #rdatatable #shiny #ggplot2
Working french NHS data #SNDS
Modelling EMR with DL-NLP approach.

[Remi Cadene @RemiCadene]

The wait is finally over!!! 😁

We just dropped an in-depth tutorial on how to build your own robot!

Teach it new skills by showing it a few moves with just a laptop.

Then watch your homemade robot act autonomously 🤯

1/🧵👇

[Andrej Karpathy @karpathy]

Nice new read on tokenization!
You've heard about the SolidGoldMagikarp token, which breaks GPT-2 because it was present in the training set of the Tokenizer, but not the LLM later.

This paper digs in in a lot more depth and detail, on a lot more models, discovering a less extreme version of the above - partially-trained tokens in both open/closed models. You have to be careful with a lot of small details and implications - weight sharing, constants in residual streams, weight-decays, regex splitting patterns, BPE, UTF-8, etc.

TLDR Tokenization remains a major pain and a large LLM attack surface. Including these partially-trained tokens in your prompts drifts the model out of distribution into undefined regions of the dynamics, areas that the model is not used to. They confuse the LLM. The paper's focus is discovery and not engineering, but it seems likely one can find "token attacks" that reliably induce target weirdness: pop-off safety, alter personality or behaviors (?), any other kind of ... otherwise undefined behavior, whatever that may look like.

Now go ask GPT-4 about _ForCanBeConverted, $PostalCodesNL, useRalative, and _typingsJapgolly :)
(or see Figure 4 of the paper at the very end for simple examples)

[Edward @aestheticedwar1]

Our open-source RAG app, Verba, can now run Ollama models locally to embed documents into Weaviate and generate answers without your data leaving your device! I tested it with Llama3, and it works great + super quickly on my M2; In this recording, I'm doing a little Verba-ception, asking Verba what Verba is 🤤

You can also use open-source models like Mistral, Claude, or your own custom LLMs. This will make doing RAG with sensitive data possible since you won't depend on APIs anymore. Looking forward to everyone trying it out 🤗

Update's coming next week!

[Hugging Face @huggingface]

Learn all the ways to fine-tune an LLM with this new course by @codecademy! 📖

[Andrej Karpathy @karpathy]

Day 24 of llm.c: we now do multi-GPU training, in bfloat16, with flash attention, directly in ~3000 lines of C/CUDA, and it is FAST! 🚀

We're running ~7% faster than PyTorch nightly, with no asterisks, i.e. this baseline includes all modern & standard bells-and-whistles: mixed precision training, torch compile and flash attention, and manually padding vocab. (Previous comparisons included asterisks like *only inference, or *only fp32 etc.) Compared to the current PyTorch stable release 2.3.0, llm.c is actually ~46% faster. My point in these comparisons is just to say "llm.c is fast", not to cast any shade on PyTorch. It's really amazing that PyTorch trains this fast in a fully generic way, with ability to cook up and run ~arbitrary neural networks and run them on a ton of platforms. I see the goals and pros and cons of these two projects as different, even complementary. Actually I started llm.c with my upcoming education videos in mind, to explain what PyTorch does for you under the hood.

How we got here over the last ~1.5 weeks - added:

✅ mixed precision training (bfloat16)
✅ many kernel optimizations, including e.g. a FusedClassifier that (unlike current torch.compile) does not materialize the normalized logits.
✅ flash attention (right now from cudnn)
✅ Packed128 data structure that forces the A100 to utilize 128-bit load (LDG.128) and store (STS.128) instructions.

It's now also possible to train multi-GPU - added:
✅ First version of multi-gpu training with MPI+NCCL
✅ Profiling the full training run for NVIDIA Nsight Compute
✅ PR for stage 1 of ZeRO (optimizer state sharding) merging imminently

We're still at "only" 3,000 lines of code of C/CUDA. It's getting a bit less simple, but still bit better than ~3 million. We also split off the fp32 code base into its own file, which will be pure CUDA kernels only (no cublas or cudnn or etc), and which I think would make a really nice endpoint of a CUDA course. You start with the gpt2.c pure CPU implementation, and see how fast you can make it by the end of the course on GPU, with kernels only and no dependencies.

Our goal now is to create a reliable, clean, tested, minimal, hardened and sufficiently optimized LLM stack that reproduces the GPT-2 miniseries of all model sizes, from 124M to 1.6B, directly in C/CUDA.

A lot more detail on: "State of the Union [May 3, 2024]"
https://t.co/eDgbngHrZ9

[Antoine Augusti @AntoineAugusti]

LLaMandement, un LLM de la @dgfip_officiel pour résumer des propositions législatives https://t.co/k5X39ot0LF

[Alise Fox @hi_im_alise]

Today I realised I can add emojis to my #Rstats code to help my squiggly brain navigate it! Now I just have to look for the correctly coloured thing instead of a wall of monochromatic text. Look at how tidy and cute my @rstudio document outline is!

[Jeremy Howard @jeremyphoward]

Big release of fastai today - @huggingface Accelerate is now supported for distributed training thanks to @TheZachMueller and @GuggerSylvain. That means you can now do distributed training in a notebook! 1/🧵

[François Chollet @fchollet]

As they say, if you don't schedule maintenance for your car, it will schedule it for you. Applies to people too. Discipline your life or your life will discipline you.

[Philéas Condemine @phileas_c]

@rstudio Waow ! Looks great 🤩

[Posit PBC @posit_pbc]

Announcing vetiver! 🏺

vetiver is a framework for MLOps tasks with a consistent approach for R and Python. Bilingual data science teams can use vetiver to version, share, deploy, and monitor trained models.

Learn more in the blog post!

https://t.co/y9HYQV3fWP

[OpenStreetMap @openstreetmap]

@premiumbusiness We are largest open source map data project globally. Map data provider for Facebook, Apple, Microsoft, Amazon Logistics, Uber, Craigslist, Snapchat, Wikimedia Maps… We are Authentic, Notable and an Active organization. Let us get this fixed. DMs open.

[Posit PBC @posit_pbc]

Introducing real-time collaborative editing on RStudio Cloud! 🎉

For Premium, Instructor, or Organization account holders, up to 5 users can access a project at the same time. Users can edit code & immediately see changes made by others.

Learn more: https://t.co/Bl8oCGghwt

[Thomas Simonini @ThomasSimonini]

We’re launching a free Deep Reinforcement Learning class on @huggingface 🥳!

Come learn with hands-on on unique environments. And publish in one line of code your trained agents on the Hub 🔥

Share with your friends who want to learn🤗!

👉 Register here https://t.co/FIsbch7qp4

[Germain Forestier @gforestier]

Changement important dans ma vie pro, à partir d'aujourd'hui je rejoins le cabinet @McKinsey en tant que consultant visualisation de données afin de produire des PowerPoint à forte valeur ajoutée. Merci à tous pour votre confiance durant ces nombreux mois, prenez soin de vous 😘

[Philéas Condemine @phileas_c]

@ChristophMolnar Awesome ! Have you added something about Integrated Gradients ? What's your opinion about this technique for neuralnets interpretation in both computer vision and natural language processing contexts ? Compared to SHAP for instance ?

[Cédric Scherer @CedScherer]

Bad pie chart but a fun bad pie chart.

[Mark Tenenholtz @marktenenholtz]

When you learn new concepts in machine learning, try to understand them on an intuitive level.

This blog from Distill is what made Momentum in SGD *click* for me.

It's an interactive demo that allows you to see the effect of different hyperparameters on the outcome!

[Gael Varoquaux 🦋 @GaelVaroquaux]

Feeling flattered that a big company like Facebook releases a library citing inspiration from joblib (https://t.co/Q3h6a7q9rw), a library powered by a tiny team on top of other commitments.

[Hadley Wickham @hadleywickham]

I'm working on a revamp of tidyr::separate() and tidyr::extract() and I'd love to see examples where you've struggled to extract multiple variables out of a complex string variable 😀