Manthan Jain

[Manthan Jain @Manthanjain110]

@cneuralnetwork I like distributed systems, infra, and call of duty

[Anirudh Sharma @anirudhology]

https://t.co/iX28n5Lc0y

[Michael Truell @mntruell]

https://t.co/jZKOk8RAsV

[Rohan Paul @rohanpaul_ai]

Terence Tao says the math behind today’s LLMs is actually simple. Training and running them mostly uses linear algebra, matrix multiplication, and a bit of calculus, material an undergraduate can handle. We understand how to build and operate these models.

The real mystery is why they work so well on some tasks and fail on others, and why we cannot predict that in advance. We lack good rules for forecasting performance across tasks, so progress is largely empirical.

A key reason is the nature of real-world data. Pure noise is well understood, perfectly structured data is well understood, but natural text sits in between, partly structured and partly random. Mathematics for that middle regime is thin, similar to how physics struggles at meso-scales between atoms and continua.

Because of this gap, we can describe the mechanisms but cannot yet explain capability jumps or give reliable task-level predictions. That mismatch, simple machinery versus hard-to-predict behavior, is the core puzzle.

----

Video from 'Dr Brian Keating' YT Channel (Link in comment)

[Manthan Jain @Manthanjain110]

Fascinating to see ML playing a crucial role in predicting how to allocate resources efficiently

[Google Research @GoogleResearch]

Scheduling virtual machines (VMs) is like a puzzle game with disappearing pieces. Here we present LAVA, an AI-based scheduling algorithm that continuously re-predicts & adapts to the lifetimes of VMs to optimize resource efficiency in cloud data centers. https://t.co/EUwx1fKZcO

[arnav @arnav_kumar]

Word..

[Raghu @IndiaTales7]

Explore Incredible Features You Might Not Have Known That Exist in Japan~A Thread 🧵

1. Moments that don't look real but really are.

[moe @mohbibi_]

if you want to understand how computers work at the hardware level.

I've seen the first video banger so far.

[Supabase @supabase]

fake news.

[Reads with Ravi @readswithravi]

Paul Graham on how writing enhances your thinking.

[Compounding Quality @QCompounding]

True power:

[Abhinav Upadhyay @abhi9u]

I found two optimizations that the CPython has done to improve the performance of its bytecode interpreter and to circumvent the cost of wrong branch prediction when executing bytecode.

Every bytecode interpreter (VM) is implemented using a giant switch case inside a loop. The switch statement checks the value of the current opcode pointed to by the instruction pointer and based on its value, the right case block executes. It looks like this:

CPython also has a similar code (but in C). The bytecode interpreter is where your Python code is really executing and as such this loop is really hot and it should run as fast as possible.

But switch statement is a branching construct. Based on the value of the opcode, the CPU needs to jump to the correct label/case block which handles that opcode.

The CPU's branch predictor will try to guess where it should jump and try to execute that path ahead of time to improve instruction throughput. However, CPython has 200+ opcodes, and usually there is not enough pattern for the CPU to be able to learn to predict the next opcode reliably. So, most of the times the branch predictor will be wrong and the bytecode execution will get even slower due to this.

One of the optimization tricks that CPython did, way back in 2003, was a hint to the CPU in the interpreter code itself.

The hypothesis is that certain opcode tend to occur in pairs, such as COMPARE_OP is usually followed by JUMP_IF_TRUE or JUMP_IF_FALSE. For such cases, they added a macro called PREDICT which was essentially predicting the next opcode (e.g. next_opcode == JUMP_IF_TRUE).

By doing so, they were priming the branch predictor. This would bring the next opcode value in the register, and also if the interpreter's prediction was right then the CPU would also make the same guess when evaluating the switch condition. This was added by @raymondh back in 2003. This was removed recently last year because of computed gotos (which is the next topic below).

The next trick is a compiler extension called computed gotos which is supported by GCC, and clang. It allows you to compute addresses of labels in your code, and as a result you can build a static table of all the labels where you handle each opcode. Then in the interpreter loop you just need to lookup the right label address in the table and jump to it (i.e. use goto).

This eliminates the branch prediction altogether because there is no branching anymore.

while (<condition>) {
char opcode = // get next opcode value;
goto jump_table[opcode];
};

This was added in 2009 (I think). When computed gotos is in use then there is no use of trying to predict the next opcode and it is disabled. Compared to the switch case, this seems to offer 15-20% speed up.

I've talked about and explained in my recent live session on CPython VM internals.

[codedamn @codedamncom]

The best way to master anything is through rigorous practice. At codedamn, we are strong believers of learning by doing.

To help you become a master at coding, we're launching 500+ real world coding problems for you to practice 🔥

RT + comment below to get early access!

[Manthan Jain @Manthanjain110]

@codedamncom Let’s go!

[All The Right Movies @ATRightMovies]

Take 5 minutes to read maybe the greatest celbrity story ever. Told by Marc Haynes.

[Manthan Jain @Manthanjain110]

Shocked to see how travelling has also dropped in the happiness index.

[Kpaxs @Kpaxs]

Social media scores as the single lowest happiness-producing leisure activity.

[anshul gupta @anshgupta64]

After the crisis in Sri Lanka’s economy, RBI did an analysis on the fiscal vulnerability of Indian states.

[Joe Pompliano @JoePompliano]

The Jordan Brand just crossed $5 billion in annual revenue for the first time.

That means Michael Jordan made $150M+ from Nike last year alone—or nearly 2x his career NBA earnings.

The part you didn't know?

Without this man, it would've never happened.

Here's the story 👇

[Ryan Holiday @RyanHoliday]

"The most important decision you make is to be in a good mood." — Voltaire