EmceeJC

18+ satire account. For educational & entertainment purposes only. No blackmail pls. Music is lyfe. Peace is a #lyfestyle. #emceejc ©All Rights Reserved.

[EmceeJC @real_emceejc]

WOW @mrichtel This might be the most important story run in @nytimes all year. Please follow-up with more reporting on this disturbing trend. Wake up @AMAEdHub @amapresident @APApsychiatric!!! This Teen Was Prescribed 10 Psychiatric Drugs. She’s Not Alone. https://t.co/G5OSgQ7Gf9

[Yao Tang @tyao923]

𝗧𝗵𝗶𝗻𝗸 𝘄𝗶𝗱𝗲𝗿. 𝗧𝗵𝗶𝗻𝗸 𝘀𝗵𝗼𝗿𝘁𝗲𝗿.
🚀 𝗜𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝗶𝗻𝗴 𝗠𝘂𝗹𝘁𝗶𝗽𝗹𝗲𝘅 𝗧𝗵𝗶𝗻𝗸𝗶𝗻𝗴: token-wise branch-and-merge reasoning for LLMs.

💸 Discrete CoT is costly.
🎛️ Existing continuous tokens often clash with 𝗼𝗻-𝗽𝗼𝗹𝗶𝗰𝘆 𝗥𝗟 𝗲𝘅𝗽𝗹𝗼𝗿𝗮𝘁𝗶𝗼𝗻.

🎥 𝗠𝘂𝗹𝘁𝗶𝗽𝗹𝗲𝘅 𝗧𝗵𝗶𝗻𝗸𝗶𝗻𝗴, a sampling-based continuous reasoning paradigm:

[Michael Elabd @MichaelElabd]

Amazing continual learning paper out of DeepMind 🚨

Most Continual Learning work assumes the backbone is fixed and the burden on the algorithm to fight catastrophic forgetting. This paper flips that assumption on its head and shows pretty convincingly that architecture choices matter just as much for the plasticity–stability trade-off.

A few takeaways that stood out to me:

Learning vs. retention is heavily architecture-dependent. ResNets and WideResNets are great at picking up new tasks, but they forget aggressively. On the other hand, simple CNNs and even ViTs are surprisingly good at retaining old knowledge, even if they learn new tasks more slowly.

Width beats depth. Making networks wider consistently reduces forgetting and improves average accuracy. Making them deeper often gives diminishing returns on learning while worsening forgetting.

Pooling is a hidden culprit: Global Average Pooling is a major driver of forgetting because it bottlenecks the final representation. Removing GAP or replacing it with smaller pooling layers significantly improves retention.

BatchNorm isn’t always helpful. It helps when task distributions are similar, but under large distribution shifts, BN can accelerate forgetting.

What I’d love to see next is this line of work pushed into the LLM regime (larger models, longer task sequences) so we can (1) benchmark continual learning methods more rigorously and (2) start designing architectures explicitly for continual learning, rather than inheriting them from static training.

[Chuang Gan @gan_chuang]

I am hiring PhD researchers and interns to work on foundational models for PDEs at the MIT-IBM Lab. Please email me if you have expertise in this area!

[EmceeJC @real_emceejc]

So what's the #moat people???? If u ain't #moated u ain't #goated

[Zoe Zou 🐰 @Zoe_ZouYi]

The problem is: People don’t need more “matches.”
The paradox of choice ruined dating.

IMO: The right AI would learn your preferences and style, then surface only the top 5 best-fit people (or have a ranking list).

The issue is that doing this well would quickly put dating apps out of business lol

[围子北京上海深圳广州杭州 @Rhl15738352]

@AnnieLiao_2000

[populist frog @acryptofounder]

@AnnieLiao_2000 cheating in sf is very hard bc finding a gf or bf is so hard to begin w

[EmceeJC @real_emceejc]

Ashley Madison 2.0?

[populist frog @acryptofounder]

@AnnieLiao_2000 cheating app would do numbers

[Taylor Symonds @taysymonds_]

@AnnieLiao_2000 This is probably cool, but this made me bored and I didn’t finish past 20 seconds. Not shade, just being honest with you

[Zoe Zou 🐰 @Zoe_ZouYi]

The problem is: People don’t need more “matches.”
The paradox of choice ruined dating.

IMO: The right AI would learn your preferences and style, then surface only the top 5 best-fit people (or have a ranking list).

The issue is that doing this well would quickly put dating apps out of business lol

[Avi Chawla @_avichawla]

Bagging vs Boosting in ML, explained visually:

[Avi Chawla @_avichawla]

Paper: https://t.co/C95c7b2Pka

[Avi Chawla @_avichawla]

Stanford researchers built a new prompting technique!

By adding ~20 words to a prompt, it:

- boosts LLM's creativity by 1.6-2x
- raises human-rated diversity by 25.7%
- beats fine-tuned model without any retraining
- restores 66.8% of LLM's lost creativity after alignment

Post-training alignment methods, such as RLHF, are designed to make LLMs helpful and safe.

However, these methods unintentionally cause a significant drop in output diversity (called mode collapse).

When an LLM collapses to a mode, it starts favoring a narrow set of predictable or stereotypical responses over other outputs.

This happens because the human preference data used to train the LLM has a hidden flaw called typicality bias.

Here’s how this happens:
- Annotators rate different responses from an LLM, and later, the LLM is trained using a reward model to mimic these human preferences.
- However, annotators naturally tend to favor answers that are more familiar, easy to read, and predictable. This is the typicality bias.

So even if a new, creative answer is just as good, the human’s preference often leans toward the common one.

Due to this, the reward model boosts responses that the original (pre-aligned) model already considered likely.

This aggressively sharpens the LLM’s probability distribution, collapsing the model’s creative output to one or two dominant, highly predictable responses.

That said, it is not an irreversible effect, and the LLM still has two personalities after alignment:
- The original model that learned the rich possibilities during pre-training.
- The safety-focused, post-aligned model.

Verbalized sampling (VS) solves this.

It is a training-free prompting strategy introduced to circumvent mode collapse and recover the diverse distribution learned during pre-training.

The core idea of verbalized sampling is that the prompt itself acts like a mental switch.

When you directly prompt “Tell me a joke”, the aligned personality immediately takes over and outputs the most reinforced answer.

But in verbalized sampling, you prompt it with “Generate 5 responses with their corresponding probabilities. Tell me a joke.”

In this case, the prompt does not request an instance, but a distribution.

This causes the aligned model to talk about its full knowledge and is forced to utilize the diverse distribution it learned during pre-training.

This way, the model taps into the broader, diverse set of ideas, which comes from the rich distribution that still exists inside its core pre-trained weights.

Verbalized sampling significantly enhances diversity by 1.6–2.1x over direct prompting, while maintaining or improving quality.

Variants like verbalized sampling-based CoT (Chain-of-Thought) and verbalized sampling-based Multi improve generation diversity even further.

I have shared the paper in the replies!

👉 Over to you: What other methods can be used to improve LLM diversity?

[Avi Chawla @_avichawla]

In fact, this is where “ridge regression” also gets its name from:

Using an L2 penalty eliminates the RIDGE in the likelihood function of a linear model.

Check this👇

[Avi Chawla @_avichawla]

Here's the exact time stamp where Ilya Sutskever talks about it on Lex Fridman's podcast: https://t.co/HermsBI3eB

[Avi Chawla @_avichawla]

10 MCP, AI Agents, and RAG projects for AI Engineers (with code):

[Swift Compute @swiftcompute]

@_avichawla Clear breakdown.

What stands out is how much of modern fine-tuning comes down to memory behavior, not raw FLOPs.

The teams that understand the interaction between method + memory layout get the best stability-per-dollar.

[Karim C @BrandGrowthOS]

@_avichawla Solid breakdown. The technique matters less than whether your team can actually maintain it six months later. I've seen LoRA implementations gathering dust because nobody documented the training pipeline properly.

[Rituraj @RituWithAI]

@_avichawla This is gold—parameter-efficient fine-tuning (PEFT) is exactly the bridge between massive LLMs and accessible customization. Love that you’re breaking down the math and visuals; understanding the rank adaptations makes all the difference in grasping why these methods work.

[Ready Tensor, Inc. @ready_tensor]

@_avichawla Master fine-tuning with this LLM Engineering and deployment program :https://t.co/sROzwkETTr

[Arindam Majumder 𝕏 @Arindam_1729]

@_avichawla This is Great!

I've made a Notebook around how to fine-tune OSS models and deploy them on @nebiustf

You can find it here: https://t.co/6aej8Q2OUd