Soham Lakhote

Ph.D. student at IISC Bangalore. Working in the field of FMCW Radar-on-chip. Interests: Semiconductors, AI Hardware, Analog/RF Circuits, Signal Processing.

[Soham Lakhote @LTI_System]

@RobertGreene @StevenBartlett But you just said in one of your previous post that accept your limitations. Why such a contradiction?

[Soham Lakhote @LTI_System]

@RobertGreene What a shitty post. I used to admire you but this post made me hate you. Accept your limitations? I have worked so hard to overcome my limitations. Have literally rewired my brain to get pleasures from doing hard things. People can overcome adversity and your post isn't helping.

[Markos @MarkosAAIG]

I’m laughing here with @jukan05 because this dropping shows exactly who understands the memory system and who don’t. Probably the same people who constantly only scream micron, SK have a low forward PE

TurboQuant lowers cost per token. and expands the context window so you can have more KV cache per request. This also leads to more total memory consumed per session. You hear that? Per session. This leads to more memory bandwidth demand.

So the whole idea is that lower cost leads to more usage, not less. So you compress the KV cache, expand the context, increase the throughput, so you can push more data through the system. That’s what it’s doing.

$SSNLF $GOOGL $NVDA $MU $000660.KS

[Soham Lakhote @LTI_System]

@damnang2 @NuttyCLD, @vikramskr and @PhotonCap would love to get your views on the Lace Lithography.

[Soham Lakhote @LTI_System]

Sounds similar to E-beam lithography. Are you really going to finish the wafer by fabricating the features point-by-point across the entire wafer? Also, I went through the website and didn't find resources on how they plan to address the exposure spread caused by e-scattering.

[Steve Jurvetson @FutureJurvetson]

Introducing Ⓛ 𝗟𝗔𝗖𝗘 𝗟𝗜𝗧𝗛𝗢𝗚𝗥𝗔𝗣𝗛𝗬
A novel approach to chip-making that can extend Moore's Law 10x beyond what is possible with light — to atomic resolution.

News today: "Manufacturers use light-based lithography systems made by the Dutch company ASML, which dominates ​the market. Lace has developed a new approach. Instead of ​light, Lace's engineers have made a form of lithography that uses a helium atom beam. With that, the Norwegian company will be able to create chip designs that are 10 times as small as what is currently possible"

"The main advantage of the helium atom beam is the industry could create features such as transistors, ‌the ⁠building blocks of modern chips, an order of magnitude smaller to an "almost unimaginable" degree, according to John Petersen, Scientific Director of Lithography at Imec, a research and innovation hub for the chip industry.

The beam Lace will use to make chips is about the width of a single hydrogen atom, or 0.1 nanometer. ASML's lithography tools use ​a beam of light that ​is about 13.5 nanometers; ⁠a human hair is about 100,000 nanometers wide.

Smaller transistors and other features would give chipmakers the ability to ramp up the performance of advanced AI processors well beyond ​the current capabilities. Lace's technology would enable chip manufacturers to print wafers at ​what is "ultimately atomic ⁠resolution" — https://t.co/q5Oh25EdUc

Now hiring in Bergen and Barcelona: https://t.co/IIERURA0lY

[Gaurab Chakrabarti @Gaurab]

The transistor, Unix, nylon, Teflon and the laser all have one thing in common: They were a result of the golden age of corporate R&D. In 1985 IBM had 400,000 employees but only 8 called "Wild Ducks." They could break all the rules, pull people off other projects, get budget on demand, and reported directly to the CEO. Bell Labs alone produced 11 Nobel laureates and 28,000 patents. Its budget came from American phone bills. Fortune 500 companies won 41% of America's top innovation awards in the 1970s. By 2006, that number dropped to 6%. Here's what killed American R&D:

1. The hostile takeover wave of the 1980s pushed executives toward short-term results
2. The AT&T breakup gutted Bell Labs from 26,000 to 19,000
3. Venture capital gave the best researchers a better deal than staying inside a corporation
4. Offshoring broke the feedback loop between making things and understanding them
5. Jack Welch turned GE from an industrial research company into a financial engineering shop and donated RCA's research lab to a nonprofit
6. The 2017 tax law penalized R&D spending so aggressively that some companies faced 4x higher tax bills for doing more research

Today the U.S. spends nearly $1 trillion a year on R&D, but two-thirds of it goes to incremental product improvement. The labs that built modern America are gone. I'm reverse-engineering what made them work. And what a modern skunkworks looks like.

[Soham Lakhote @LTI_System]

@NuttyCLD The biggest advantage for the scale-up is that you can work with Multi-Mode Fibres, which have a core diameter of 50 µm, so the alignment tolerance isn't very strict. However, this will be limited to scale-up applications only due to smearing issues of MMF over longer distances.

[Soham Lakhote @LTI_System]

@soicfinance Sorry, but first time I'm seeing the post written mostly by ChatGPT and not SOIC.

[Soham Lakhote @LTI_System]

@jitenkparmar Sir, at least give credit to the person whose post you stole. Original post: https://t.co/0tuUJ9Jmu6
@levelsio

[@levelsio @levelsio]

The biggest fumble in business ever might be Philips spinning off ASML, TSMC and NXP

Philips co-founded ASML in 1984, then co-founded TSMC in 1987, then they founded NXP

They sold each of them for short term profits in the 2000s

ASML is now worth $545B
TSMC is worth $1.76T
NXP is worth $50B

Philips today is worth just $27B

If they'd never sold, Philips would be the largest company in the EU today, worth $650B

Philips CEO Cor Boonstra called it "making money with the success of the past"

🤡

[Aditya Kondawar @aditya_kondawar]

The moment that would define KPR Mill's soul happened in 2001. A young mill worker, barely 18, approached Chairman K.P. Ramasamy during a factory visit. She didn't ask for a raise or better working conditions. She asked if he could help her complete her education—something poverty had forced her to abandon. Ramasamy didn't just say yes. He asked how many others wanted the same opportunity.

Within weeks, KPR had arranged a correspondence program for 50 women workers. The company partnered with Alagappa University to conduct classes on-site. Teachers came to the factory. Study materials were provided free. Work schedules were adjusted to accommodate exam preparation. The initial results stunned everyone: these women, many of whom had been out of school for years, passed their +2 examinations with distinction.

But Ramasamy saw something beyond academic success. These educated workers made fewer errors, suggested process improvements, and most remarkably, stayed with the company. In an industry where annual attrition of 30-40% was normal, KPR's educated workers showed attrition rates below 5%. The ROI was undeniable, but for Ramasamy, it was never about ROI.

The program scaled rapidly. By 2010, over 5,000 women had completed their basic education. Some went further, pursuing bachelor's degrees in commerce and arts. A few even completed MBAs while working full-time on the shop floor. The company established libraries, computer centers, and study halls within factory premises. What started as correspondence courses evolved into a full-fledged education ecosystem.

The numbers are staggering: 27,000 women have now completed their education through KPR's programs. But the real impact goes beyond diplomas. These women send their children to English-medium schools. They make financial decisions for their families. They've broken generational cycles of poverty. And 194 of them have leveraged their education to land jobs at companies like Tata Electronics, Tech Mahindra, and Titan—which Ramasamy celebrates rather than laments.

"We treat these women like we would treat our own daughters," Ramasamy often says. This isn't corporate PR speak. The company provides free transportation from 200 villages. It runs subsidized canteens serving nutritious meals. It offers free medical care, including annual health checkups. During festivals, workers receive bonuses equal to several months' salary. The company even maintains marriage halls that workers can use for free for family functions.

The economic impact of this approach is counterintuitive but powerful. KPR's labor cost as a percentage of revenue is actually lower than industry averages, despite higher per-worker spending. The reason? Productivity. Educated workers produce 20-30% more output per shift. Quality defect rates are 50% lower. The need for supervision is minimal. Workers train each other, creating a self-reinforcing culture of excellence.

This is not a company, it is a university indeed!

Src – Empor top, no reco

[Gauri Gupta @gauri__gupta]

Published my personal notes from interview prep at top AI labs like OpenAI, Anthropic, WorldLabs, etc, revisiting core concepts in large-scale ML design and optimization for efficient model training and inference.

Hope it’s useful to others exploring and deep-diving into ML/LLMs optimization techniques.
https://t.co/amZqdwgLMo

[Fundoo Professor @Sanjay__Bakshi]

What a fascinating way to teach!

[Andrew Côté @Andercot]

I used to work 6am - 6pm, 6 days a week, on a construction site in my early 20s.

Honestly? It fucking sucked, dude. I would sit in my car outside the site at 530am, desperately drinking a coffee, telling myself over and over again, "god I wish I was in sciences"

Because every night when I got home what did I do? Watch Walter Levin MIT Open Courseware physics lectures. I had already exhausted all the popular science books long before so just started on undergrad level physics. The alternative was drinking a six pack of beer like everyone else and watching bullshit TV.

The construction site job was actually better than what I was doing before. Landscaping, stone masonry shit. Backbreaking labor, truly. Breaking concrete slabs up with a sledgehammer and carrying bricks all day. That's literally a punishment in prison.

There was a company event for the property development Corp doing the construction I was working for where everyone talked about their degrees. Most people had been at the company for almost a decade, did random unrelated degrees.

I realized. If I didn't take control of my life the years would tick by. So I went back to school for engineering physics at age, like, 25. I probably wouldn't graduate until I was 30, but shit.

You're going to turn 30 one day anyway. Might as well be doing something you chose.

A year into schooling I had my first paying job in a physics lab, basically minimum wage, but my god. I was getting paid to work in a physics lab. I could drink coffee and read papers, build cool stuff. It was insane.

The kids around me had no idea how lucky we were to be there. They hadn't suffered being trapped in dead end jobs that leave you too exhausted to really think, plan, get ahead. So I viciously worked my ass off through out engineering physics to play the game as best I could. Get the best internships, connections, etc. By the end of undergrad I was taking graduate level classes and outperforming the PhD students at them.

Everything since then had gone better than I could've imagined. I used to think - wow, the dream would be designing fusion reactors, if only. Now I have patents in fusion reactor design. I've worked on particle accelerators, LEO satellite communications, beam driven fusion devices, finite element analysis for RF source design at SLAC.

So no. Fuck mind breaking manual labor. Leave it for the robots. Choose your own path.

I will say though. There are few things as therapeutic and full body workout as shoveling sand. I can show you at least a dozen different sand shoveling techniques to work every muscle in your upper body. Also wheelbarrow technique.

[Andrej Karpathy @karpathy]

Finally had a chance to listen through this pod with Sutton, which was interesting and amusing.

As background, Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea is sufficiently "bitter lesson pilled" (meaning arranged so that it benefits from added computation for free) as a proxy for whether it's going to work or worth even pursuing. The underlying assumption being that LLMs are of course highly "bitter lesson pilled" indeed, just look at LLM scaling laws where if you put compute on the x-axis, number go up and to the right. So it's amusing to see that Sutton, the author of the post, is not so sure that LLMs are "bitter lesson pilled" at all. They are trained on giant datasets of fundamentally human data, which is both 1) human generated and 2) finite. What do you do when you run out? How do you prevent a human bias? So there you have it, bitter lesson pilled LLM researchers taken down by the author of the bitter lesson - rough!

In some sense, Dwarkesh (who represents the LLM researchers viewpoint in the pod) and Sutton are slightly speaking past each other because Sutton has a very different architecture in mind and LLMs break a lot of its principles. He calls himself a "classicist" and evokes the original concept of Alan Turing of building a "child machine" - a system capable of learning through experience by dynamically interacting with the world. There's no giant pretraining stage of imitating internet webpages. There's also no supervised finetuning, which he points out is absent in the animal kingdom (it's a subtle point but Sutton is right in the strong sense: animals may of course observe demonstrations, but their actions are not directly forced/"teleoperated" by other animals). Another important note he makes is that even if you just treat pretraining as an initialization of a prior before you finetune with reinforcement learning, Sutton sees the approach as tainted with human bias and fundamentally off course, a bit like when AlphaZero (which has never seen human games of Go) beats AlphaGo (which initializes from them). In Sutton's world view, all there is is an interaction with a world via reinforcement learning, where the reward functions are partially environment specific, but also intrinsically motivated, e.g. "fun", "curiosity", and related to the quality of the prediction in your world model. And the agent is always learning at test time by default, it's not trained once and then deployed thereafter. Overall, Sutton is a lot more interested in what we have common with the animal kingdom instead of what differentiates us. "If we understood a squirrel, we'd be almost done".

As for my take...

First, I should say that I think Sutton was a great guest for the pod and I like that the AI field maintains entropy of thought and that not everyone is exploiting the next local iteration LLMs. AI has gone through too many discrete transitions of the dominant approach to lose that. And I also think that his criticism of LLMs as not bitter lesson pilled is not inadequate. Frontier LLMs are now highly complex artifacts with a lot of humanness involved at all the stages - the foundation (the pretraining data) is all human text, the finetuning data is human and curated, the reinforcement learning environment mixture is tuned by human engineers. We do not in fact have an actual, single, clean, actually bitter lesson pilled, "turn the crank" algorithm that you could unleash upon the world and see it learn automatically from experience alone.

Does such an algorithm even exist? Finding it would of course be a huge AI breakthrough. Two "example proofs" are commonly offered to argue that such a thing is possible. The first example is the success of AlphaZero learning to play Go completely from scratch with no human supervision whatsoever. But the game of Go is clearly such a simple, closed, environment that it's difficult to see the analogous formulation in the messiness of reality. I love Go, but algorithmically and categorically, it is essentially a harder version of tic tac toe. The second example is that of animals, like squirrels. And here, personally, I am also quite hesitant whether it's appropriate because animals arise by a very different computational process and via different constraints than what we have practically available to us in the industry. Animal brains are nowhere near the blank slate they appear to be at birth. First, a lot of what is commonly attributed to "learning" is imo a lot more "maturation". And second, even that which clearly is "learning" and not maturation is a lot more "finetuning" on top of something clearly powerful and preexisting. Example. A baby zebra is born and within a few dozen minutes it can run around the savannah and follow its mother. This is a highly complex sensory-motor task and there is no way in my mind that this is achieved from scratch, tabula rasa. The brains of animals and the billions of parameters within have a powerful initialization encoded in the ATCGs of their DNA, trained via the "outer loop" optimization in the course of evolution. If the baby zebra spasmed its muscles around at random as a reinforcement learning policy would have you do at initialization, it wouldn't get very far at all. Similarly, our AIs now also have neural networks with billions of parameters. These parameters need their own rich, high information density supervision signal. We are not going to re-run evolution. But we do have mountains of internet documents. Yes it is basically supervised learning that is ~absent in the animal kingdom. But it is a way to practically gather enough soft constraints over billions of parameters, to try to get to a point where you're not starting from scratch. TLDR: Pretraining is our crappy evolution. It is one candidate solution to the cold start problem, to be followed later by finetuning on tasks that look more correct, e.g. within the reinforcement learning framework, as state of the art frontier LLM labs now do pervasively.

I still think it is worth to be inspired by animals. I think there are multiple powerful ideas that LLM agents are algorithmically missing that can still be adapted from animal intelligence. And I still think the bitter lesson is correct, but I see it more as something platonic to pursue, not necessarily to reach, in our real world and practically speaking. And I say both of these with double digit percent uncertainty and cheer the work of those who disagree, especially those a lot more ambitious bitter lesson wise.

So that brings us to where we are. Stated plainly, today's frontier LLM research is not about building animals. It is about summoning ghosts. You can think of ghosts as a fundamentally different kind of point in the space of possible intelligences. They are muddled by humanity. Thoroughly engineered by it. They are these imperfect replicas, a kind of statistical distillation of humanity's documents with some sprinkle on top. They are not platonically bitter lesson pilled, but they are perhaps "practically" bitter lesson pilled, at least compared to a lot of what came before. It seems possibly to me that over time, we can further finetune our ghosts more and more in the direction of animals; That it's not so much a fundamental incompatibility but a matter of initialization in the intelligence space. But it's also quite possible that they diverge even further and end up permanently different, un-animal-like, but still incredibly helpful and properly world-altering. It's possible that ghosts:animals :: planes:birds.

Anyway, in summary, overall and actionably, I think this pod is solid "real talk" from Sutton to the frontier LLM researchers, who might be gear shifted a little too much in the exploit mode. Probably we are still not sufficiently bitter lesson pilled and there is a very good chance of more powerful ideas and paradigms, other than exhaustive benchbuilding and benchmaxxing. And animals might be a good source of inspiration. Intrinsic motivation, fun, curiosity, empowerment, multi-agent self-play, culture. Use your imagination.

[Soham Lakhote @LTI_System]

@ST_PYI Most of the senior people in the industry who have moved up to mouth watering positions have some work experience abroad before they moved back here for work.

[Soham Lakhote @LTI_System]

@ST_PYI Sorry, but this is definitely a bad advice. We don't have research infrastructure here. IITs and IISc prefer to hire professors who have good research/industry experience abroad. Universities are far better in terms of research and teaching abroad.

[Amaan @amaan8429]

I am happy that people are finally realizing that they have been fooled💀💀

[Soham Lakhote @LTI_System]

@DeepikaBhardwaj @smileindiatrst You are a genuine and pure soul.

[𝕲𝖆𝖓𝖊𝖘𝖍 * @ggganeshh]

Hiranandani ji talking about why real estate has become unaffordable

1. Ready reckoner rates purposely kept high
2. Tax ter'ism taking away 50% of your money
3. Revdi houses

[Soham Lakhote @LTI_System]

@mishrag47 @DeepikaBhardwaj