Ramesh Sampath

ML Engineer. Previously @landingAI @lilyai_ @fivestars @insightfellows

[Ramesh Sampath @sampathweb]

This is the quote I come back to for motivation when I am not happy with what I am making.
- https://t.co/YYODoFy5PR
- https://t.co/0jzaXawlZs

Thank you @iraglass

[Simon Willison @simonw]

Here's the table of contents for my end-of-year review of things we learned out about LLMs in 2024 - we learned a LOT

[Vala Afshar @ValaAfshar]

Life lessons from @rogerfederer (must watch)

1 Effortless is a myth
2 Belief in yourself has to be earned
3 Grit > Gift
4 Discipline is talent
5 Trust and loving the process is talent
6 You can do your best and still lose
7 Life is bigger than the court

[Martin Görner @martin_gorner]

New Keras starter notebook from @awsaf49 for the "LMSYS Chatbot Arena Human Preference" competition on Kaggle. This one is interesting for how it achieves preference classification over pairs of (prompt+response) using the DeBERTaV3 model from KerasNLP.
https://t.co/EVcSlhndJx

[François Chollet @fchollet]

If you missed it, this I/O session on LLMs with Keras 3 is a great tutorial on LLM training and fine-tuning best practices https://t.co/QwktS21G9E

[Rama Narasimhan @rama41296]

Presented our team project "CityHub" in the
@Meta #Llama3Hackathon at @SHACK15sf.

It is an AI-powered conversational assistant that provides personalized information on San Francisco city services, aiding residents and visitors in navigating government websites.

[Martin Görner @martin_gorner]

New Kaggle starter notebook from @awsaf49 for the Automated Essay Scoring competition: https://t.co/3G3ROmrQw6.
It showcases the right way to do ordinal regression in Keras, i.e. how to predict ordered integer grades reliably (no it's neither a classification, nor regression!)

[Kaggle @kaggle]

🎥 Check out the details of our recent Keras integration with @fchollet and @MeganRisdal!

[Ramesh Sampath @sampathweb]

Keras 3 makes switching DL frameworks a env. variable setting and the rest of the code works the same. Nice example using LM and StableDiffusion. Excited to try it out.

[Martin Görner @martin_gorner]

Good looking Jax coders!
Seriously, this is a Keras 3.0 demo, runnig KerasCV and KerasNLP generative models on Jax, TensorFlow or PyTorch.
https://t.co/4rzyoepaWM
It also shows how to install everything.

[Ramesh Sampath @sampathweb]

Excited to have framework optionality with the familiar Keras API, where simple usecases are easy and complex usecases are made possible!

[François Chollet @fchollet]

Big news: we just released Keras 3.0!

▶ Run Keras on top of JAX, TensorFlow, and PyTorch
▶ Train faster with XLA compilation
▶ Unlock training runs with any number of devices & hosts via the new Keras distribution API

It's live on PyPI now! 🚀

[Vipul Vaibhaw @vaibhaw_vipul]

Donald Knuth advice to young engineers -
- Follow your own instincts.
- Prestige comes from doing good science, not just popular science.
- Deep knowledge builds confidence.
- A liberal education involves learning everything about something and something about everything.

[Sanyam Bhutani @bhutanisanyam1]

How to become an expert at any thing 🙏

I rediscovered this gem by @karpathy in my bookmarks today

[Bindu Reddy @bindureddy]

Large Language Models Explained - At a High Level

LLMs such as GPT-4, Bard and LLama appear magical and have displayed emergent human-like intelligence

Fundamentally, the work on a pretty simple principle - next-word prediction.

An LLM, uses something called conditional probability to predict the next word in a sequence. Simply put, that's the likelihood of an event happening, given that another event has already happened. For our model, it's the probability of a particular word following a given sequence of words.

Imagine the sequence of words is "I like to eat...". The AI model calculates the probability of every word it knows coming after that sequence. "Pizza", "cake", "vegetables", "tires"... everything. The word with the highest probability is what it guesses.

How does the model get these probabilities? From the training data. These models trained on massive amounts of training data and it learns the probabilities of word sequences from this data. Every time it sees "I like to eat pizza", it tweaks the conditional probability of "pizza" following "I like to eat" upwards.

This training is done through a process called optimization, where the error between its predictions and the actual next word in the training data is minimized.

The process produces the base model, over which fine-tuning techniques called RLHF are further applied to align and refine the model's answers

It's still quite remarkable that such a simple and elegant technique can exhibit such delightful behavior! Having said that, this is a oversimplification and there are a lot more technical details to training LLMs.

I recommend starting here -

Blog and pic credit - https://t.co/bqubrsMliX
YT video- https://t.co/17oaXcfS4r

[Ramesh Sampath @sampathweb]

The Question You Must Ask Yourself 🤔🎯 https://t.co/ztNYw5DL1P via @YouTube

[François Chollet @fchollet]

We're launching Keras Core, a new library that brings the Keras API to JAX and PyTorch in addition to TensorFlow.

It enables you to write cross-framework deep learning components and to benefit from the best that each framework has to offer.

Read more: https://t.co/xmmxBfSZgh

[Geoffrey Hinton @geoffreyhinton]

The University of Toronto made a video for a non technical audience in which I explain how deep learning works, the enormous promise of this technology and some of the potential risks. https://t.co/d0Izl3RYbS

[Sanyam Bhutani @bhutanisanyam1]

Emerging LLM Application Stack! 👌

This write up by @a16z is one of the best and most pragmatic on the topic.

I would even call this a perfect complimentary to @karpathy’s State of GPT talk.

The post goes beyond just covering architectural overview to the why, alternative suggestions and opinions on what might emerge for different parts of the application pipeline, when building with LLMs.

Example: Most people assume context length will deprecate vector dbs. However, the post points out the performance and cost trade offs. It’s sprinkled with such practical nuggets.

It’s easily one of the crispiest reads on the topic and I’d recommend you must read this.

Image credit 🙏: Taken from the blog

https://t.co/UzMpnWxEUu

[Sebastian Raschka @rasbt]

GPT in 60 Lines of NumPy -- a simple yet complete technical introduction to the GPT as an educational tool.

https://t.co/zSsKcb229E

I love this! It reminds me why coding in Python can be so much fun!

[François Chollet @fchollet]

Announcement: the new Keras model saving format is available in tf-nightly.

This format is aimed at reloading the exact same Python object, in a safe way (it does not rely on bytecode or pickling at all). In the future, the TF SavedModel format will be for inference-only export.

[François Chollet @fchollet]

New tutorial on https://t.co/m6mT8SaHBD: use Textual Inversion to teach StableDiffusion about a new visual concept (such as a specific character) to generate more images featuring the same concept. https://t.co/6mpyozrPvt

[Jay Alammar @JayAlammar]

The Illustrated Stable Diffusion

https://t.co/sbjKHP4Pol

New post!

Over 30 visuals explaining how Stable Diffusion works (diffusion, latent diffusion, CLIP, and a lot more).