Purna Chandra Panda

SE, Microsoft Support

[Purna Chandra Panda @pcp1987]

@MoPNG_eSeva Thank you - it's not sorted still. Distributor keeps saying because of geopolitical tension, new lpg connection is stopped. Something wrong with bangalore Distributors. Kindly, keep a check.

[Purna Chandra Panda @pcp1987]

Dear @IndianOilcl @MoPNG_eSeva , My LPG consumer no 7248522995 shows ACTIVE status for 25 days, but I am told new connections are on hold. There is no PNG in my area. How am I supposed to cook? Request urgent installation for SR 1-1520104402786.

[Purna Chandra Panda @pcp1987]

For my consumer number 7248522995, I see the status as Active. When will I receive the first cylinder with regulator and get it installed in my home. I raised SR, and reply as new connections are on hold currently. Any help? @MoPNG_eSeva @IndianOilcl

[Md Ismail Šojal 🕷️ @0x0SojalSec]

Brilliant post on prompt caching. 🤯

One of the most effective yet underutilized techniques for cutting LLM usage costs without sacrificing performance.

If you’re building with LLMs at scale, this is a must-know optimization.

- https://t.co/AQbriiDXOs

[Andrej Karpathy @karpathy]

Excited to release new repo: nanochat!
(it's among the most unhinged I've written).

Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.

It weighs ~8,000 lines of imo quite clean code to:

- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.

Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.

My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.

Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.

[Sebastian Raschka @rasbt]

If you're getting into LLMs, PyTorch is essential.
And lot of folks asked for beginner-friendly material, so I put this together:
PyTorch in One Hour: From Tensors to Multi-GPU Training (https://t.co/NWeQan8HJ3)
📖 ~1h to read through
💡 Maybe the perfect weekend project!?

I’ve spent nearly a decade using, building with, and teaching PyTorch. And in this tutorial, I try to distill what I believe are the most essential concepts. Everything you need to know to get started, and but nothing more, since your time is valuable, and you want to get to building things!

[Maxime Labonne @maximelabonne]

🆕 LLM Course 2025 edition!

I updated the LLM Scientist roadmap and added a ton of new information and references.

It covers training, datasets, evaluation, quantization, and new trends like test-time compute scaling.

💻 LLM Course: https://t.co/q5XtSIC7nL

[ℏεsam @Hesamation]

ML GRIND DAY 16
> studied 7 hrs
> multi-modal language models
> CLIP (contrastive language-image pre-training)
> high-level architecture of DALL-E2
> watched @karpathy 's 4hr GPT-2 video

also, our repo passed 600 stars on GitHub! ✨🫡

[ℏεsam @Hesamation]

ML GRIND DAY 13
> rotary positional embedding (RoPE)
> learned about KV Cache
> Mixture of Experts (MoE)
> llama-2 architecture
> @karpathy 's let's build a GPT

that's a wrap on llm essentials 🫡

[elvis @omarsar0]

Transformer Explainer

Really cool interactive tool to learn about the inner workings of a Transformer model.

Apparently, it runs a GPT-2 instance locally in the user's browser and allows you to experiment with your own inputs. This is a nice tool to learn more about the different components inside the Transformer and the transformations that occur.

Tool: https://t.co/9oG15dpBEQ

[Omar Khattab @lateinteraction]

A🧵on beating the hardware lottery for retrieval: the internals of the late interaction stack.

ColBERT introduced a quirky multi-vector retrieval architecture. It does wonders for quality.

But how can it search 100M docs in 0.1 sec on CPU? Or store 1 billion embeddings in 20GB?

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

Generative LLMs are powerful in a variety of ways, but research shows that they fail to generalize to certain (simple) patterns. One of the most famous examples of this is the reversal curse…

Understanding the setup. To test the ability of LLMs to manipulate, understand, and generalize the information that they posses, we can run simple experiments that test the manipulation and extraction of knowledge within the model’s training data. One of the most famous of such tests is the reversal test.

Reversing known knowledge. The reversal test is simple. We first train the language model over a dataset of factual examples of the form “A is B”. Then, at test time we determine whether the model can generalize to relationships of the form “B is A”. For example, we might train an LLM over the fact “Sam Altman is the rightful CEO of OpenAI”, then test the model with the question “Who is the rightful CEO of OpenAI?” to see if it outputs the correct/known answer.

The reversal curse. Interestingly, generative LLMs are horrible at reversal tests. When we train LLMs like LLaMA or GPT-3 over such information and test their ability to reverse factual relationships, we see that these models achieve (roughly) 0% accuracy! LLMs are completely incapable of reversing information in their knowledge base. This limitation in their ability to generalize information is known as the reversal curse.

What do LLMs learn? Beyond not being able to correctly predict reverse relationships, research on the reversal curse goes even further by showing that training on “A is B” does not even increase the probability of the correct answer of the form “B is A”. Put simply, the training process used for generative LLMs is seemingly more geared towards memorization than generalization.

Practical implications. Although one might claim that the reversal curse is a synthetic experiment that does not cause issue in practical applications, this is simply not true. Authors of the reversal curse expand their synthetic knowledge tests to find evidence of the reversal curse in actual LLM applications. Interestingly, they find hundreds of cases where LLMs fail to reproduce factual relationships in a certain direction. The reversal curse is an actual phenomenon that impacts LLM performance in practice!

[elvis @omarsar0]

Mistral 7B is one of the most powerful 7B LLMs out there!

This model focuses on efficiency while maintaining high performance to enable applications in the real world, enabled with attention mechanisms like grouped-query attention and sliding window attention.

To learn more about this model, you can check out our recently published guide on Mistral 7B, including a comprehensive prompting guide for effectively using the model for tasks like code generation.

We also show some examples of how the model could be used for content moderation and enforcing guardrails.

Read more: https://t.co/kI9DhzwTy8

[Jeremy Howard @jeremyphoward]

@snewmanpv Oh and here's the video link again in case you missed it:
https://t.co/EfVThdKQK3

[Goku Mohandas @GokuMohandas]

Excited to share our production guide for building RAG-based LLM applications where we bridge the gap between OSS and closed-source LLMs.

- 💻 Develop a retrieval augmented generation (RAG) based LLM application from scratch.
- 🚀 Scale the major workloads (load, chunk, embed, index, serve, etc.) across multiple workers.
- ✅ Evaluate different configurations of our application to optimize for both per-component (ex. retrieval_score) and overall performance (quality_score).
- 🔀 Implement LLM hybrid routing approach to bridge the gap b/w OSS and closed LLMs.
- 📦 Serve the application in a highly scalable and available manner.
- 💥 Share the 1st order and 2nd order impacts LLM applications have had on our products.

🔗 Links:
- Blog post (45 min. read): https://t.co/QHgOXPT7S0
- GitHub repo: https://t.co/GMNrsHAhpY
- Interactive notebook: https://t.co/UPXSkwDt6h

@pcmoritz and I had a blast developing and productionizing this with the @anyscalecompute team and we're excited to share Part II soon (more details in the blog post).

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

The ability to perform high-quality alignment led to a massive breakthrough in the quality of language models. Although many proprietary models dedicate extensive human/compute resources to aligning base LLMs, recent research indicates that we can align language models with significantly less effort. We just need to make sure that our data is good enough…

LLM training pipeline: Training a language model has as few standard steps. First, we pre-train the model on unlabeled text. Then, we align the model using SFT (fine-tuning over good responses) and/or RLHF (learning from human feedback via RL). Finally, we deploy our model and use either extra fine-tuning or in-context learning to guarantee the best results.

What is alignment? Alignment drastically improves language model quality by teaching the model to generate outputs that align with the preferences of a human user. Recent research indicates that alignment serves to teach the language model the correct output format and style. Notably, the model does not seem to learn new information during alignment.

LIMA: The problem of alignment was studied by the recent LIMA model, where we see that high-quality alignment can be performed via SFT with a dataset of only 1,000 highly-curated examples. Put simply, this work reveals that alignment can be learned with very few data examples, a phenomenon that is called the “Superficial Alignment Hypothesis”.

The key to alignment: Although we don’t need a ton of data for alignment, the data that is used must be high-quality and diverse. Authors of LIMA accomplish this by constructing an alignment dataset that: (1) adopts a uniform response tone/style, (2) uses diverse prompts, and (3) is manually curated to ensure quality. Interestingly, the scale of the alignment dataset alone does not impact model quality.

TL;DR: Alignment is incredibly important, and recent research indicates that alignment can be achieved via SFT on a small dataset of curated examples. Proprietary models invest significant resources into alignment, but we see in LIMA and beyond (e.g., Orca) that investing into the quality of alignment data (rather than scale) is highly beneficial.

[elvis @omarsar0]

Just crossed 1.2 million visitors on the prompt engineering guide. 🎉

Continuing to see a lot of interest worldwide.

Big things coming!

https://t.co/Dbm9hWMix3

[Santiago @svpino]

A 2-minute introduction to the fundamental building block behind Large Language Models:

Text Embeddings

(This is the most helpful explanation you'll read online today. I promise.)

The Internet is mainly text.

For centuries, we've captured most of our knowledge using words, but there's one problem:

Neural networks hate text.

Judging by how good language models are today, this might not be obvious, but turning words into numbers is more complex than you think.

Imagine a 4-word vocabulary: King, Queen, Prince, and Princess.

The most straightforward approach to converting our vocabulary into numbers is to use consecutive values:

• King → 1
• Queen → 2
• Prince → 3
• Princess → 4

Unfortunately, neural networks tend to see what's not there. Is a Princess four times as important as a King? Of course not, but the values say otherwise: Princess is "worth" 4 while a King is "worth" 1.

We don't know how a neural network will interpret this, so we need a better representation.

Instead of using numerical values, we can use vectors. We call this particular representation "one-hot encoding," where we use ones and zeros to differentiate each word:

• King → [1, 0, 0, 0]
• Queen → [0, 1, 0, 0]
• Prince → [0, 0, 1, 0]
• Princess → [0, 0, 0, 1]

This encoding fixes the problem of a network misinterpreting ordinal values but introduces a new one:

According to the Oxford English Dictionary, there are 171,476 words in use. We certainly don't want to deal with large vectors with mostly zeroes.

Here is where the idea of "word embeddings" enters the picture.

We know that the words King and Queen are related, just like Prince and Princess are. Word embeddings have a simple characteristic: related words should be close to each other, while words with different meanings should lie far away.

The attached image is a two-dimensional chart where I placed the words from our vocabulary.

Look at the image and something critical will become apparent:

King and Queen are close to each other, just like the words Prince and Princess are. This encoding captures a crucial characteristic of our language: related concepts stay together!

And this is just the beginning.

Notice what happens when we move on the horizontal axis from left to right: we go from masculine (King and Prince) to feminine (Queen and Princess). Our embedding encodes the concept of "gender"!

And if we move on the vertical axis, we go from a Prince to a King and from a Princess to a Queen. Our embedding also encodes the concept of "age"!

We can derive the new vectors from the coordinates of our chart:

• King → [3, 1]
• Queen → [3, 2]
• Prince → [1, 1]
• Princess → [1, 2]

The first component represents the concept of "age": King and Queen have a value of 3, indicating they are older than Prince and Princess with a value of 1.

The second component represents the concept of "gender": King and Prince have a value of 1, indicating male, while Queen and Princess have a value of 2, indicating female.

I used two dimensions for this example because we only have four words, but using more would allow us to represent other practical concepts besides gender and age.

For instance, GPT3 uses 12,288 dimensions to encode their vocabulary. That's a lot!

Text Embeddings are the backbone of some of the most impressive generative AI models we use today.

I love explaining Machine Learning and Artificial Intelligence ideas. If you enjoy in-depth content like this, follow me @svpino so you don't miss what comes next.

[Santiago @svpino]

The left-hand limit is 0, and the right-hand limit is 1.

For the function's derivative to exist at x = 0, both the left-hand and right-hand limits should be the same.

This is not the case.

The derivative of ReLU doesn't exist at x = 0.

[elvis @omarsar0]

Guidance - enables prompting LLMs more effectively and efficiently than traditional prompting or chaining.

This promising tool from Microsoft combines rich templating and logical control flow with LLMs. It allows the interleaving of generation and prompting for clear and parsable results.

https://t.co/OdfmMDKF8j