Soumya Jain

Research Manager @ Cambridge AI Safety Hub · AI Product Manager @ Terrabase

context layers, evals, governance

studying minds, agents & other unstable systems

[Soumya Jain @wild_and_empty]

Goal for this lifetime: Fierce in discipline, soft in heart, empty of self, precise in craft.

[Soumya Jain @wild_and_empty]

@dadalife369 Dharamshala

[Soumya Jain @wild_and_empty]

I love living in the Himalayas!!!!!

[Soumya Jain @wild_and_empty]

@Bitcoinistic Stopping at 40k is a privilege really

[Soumya Jain @wild_and_empty]

you get around 80,000 working hours in one life.

using them well is an identity question. you become what you repeatedly give your attention to.

[Soumya Jain @wild_and_empty]

@Bhavipatel03 Not kangra, a lil higher up

[Soumya Jain @wild_and_empty]

@Bitcoinistic It's Dharamshala, Himachal :)

[Cas (Stephen Casper) @StephenLCasper]

Anthropic and OpenAI are publicly pointing out how having the option to slow down AI would offer a potentially critical form of optionality in the future. The correct response for any policymaker should be "Damn, this is serious. How can I help build that capacity?"

[@mrgunn ⏸️ @mrgunn]

You can almost feel the whiplash.

[Jasmine Wang @j_asminewang]

“One goal of such an organization should be to make it possible for the world to take coordinated action, including slowing frontier development when needed, so societal resilience, safety, and alignment can keep pace.”

[Brian Armstrong @brian_armstrong]

Good take

My guess is
- demand for intelligence is near infinite
- but 80% of workloads will be running on 99% cheaper models within 12-18 months
- 20% of workloads will still run on latest gen models where IQ maxing is important (scientific breakthroughs, higher level ochestrator agents?)
- rough analogy might be what % of macbooks or gaming PCs sold have the maxed out specs for CPU/GPU, prices are falling much faster than Moore's law here though
- this leads me to think the limiting factor will be energy and compute, not better models

At Coinbase we're working hard on routing prompts to cheaper models where appropriate, and in some cases have been able to keep costs roughly flat, while token usage continues to grow exponentially.

[Soumya Jain @wild_and_empty]

everyone talks about prompting as if the problem is better instructions but a lot of “intent” is actually historical context -- preferences, exceptions, tradeoffs, taste, permissions,what changed yesterday

enterprise context layers are the obvious version of this. personal context layers are next.

the weird part: as AI gets better, humans may get worse at explicitly knowing what they want because intent formation itself starts getting outsourced.

[gabriel @gabriel1]

every job will turn into explaining your intentions to ai

explaining what you want to ai is surpringly time consuming, coders already spend 80% of their time doing it, and this will be true for everyone

[Soumya Jain @wild_and_empty]

A “confused person” is not simply someone experiencing doubt or confusion. Everyone has moments of not knowing, questioning and needing to work toward clarity.

But when doubt is repeatedly taken personally like “I am confused,” “I can never know,” “I am someone who doesn’t understand”, it can harden into identity.

Then confusion stops being a temporary state and becomes a personality structure.

The healthier process is: notice the doubt, understand what exactly is unclear, stay with the process of inquiry, and let clarity emerge. Confusion is workable when it is seen as a state. It becomes binding when it is turned into self.

[Soumya Jain @wild_and_empty]

we need a much better interface between ai safety experts and policymakers, not just occasional briefings but regular high trust, low latency coordination.

the policy people need the technical threat models earlier, and the technical people need to understand policy constraints better.

[Andrea Miotti @andreamiotti]

@GFuterman At ControlAI we've talked to nearly 400 lawmakers across the US, UK, Canada and Germany in the last 18 months, plus staff in 100+ congressional offices.

For most of them, meeting with us was the first time they learned about the threat from ASI.

We need much, much more of this

[Soumya Jain @wild_and_empty]

this sounds like a really interesting direction. I’d also be curious about tracking Buddhist inspired traits like non harming, compassion, equanimity, non reification, etc.

Wouldn't be “is the model enlightened?”, but does it reduce hostility, confusion, and rigid self/other narratives across adversarial contexts? Happy to say more if useful!

[Soumya Jain @wild_and_empty]

@RogerThisdell this sounds so much fun!

[Soumya Jain @wild_and_empty]

@bakasuraptor https://t.co/qWZp39pAaW

[Soumya Jain @wild_and_empty]

People in India, this is exactly the kind of thing we need more of.

AI safety cannot remain concentrated only in the US/UK/EU. As AI systems get deployed globally, India and the Global South need more people thinking seriously about safety, evaluation, governance, misuse and real world deployment risks.

Apart Research is hosting an in-person Global South AI Safety Hackathon in Delhi and Bangalore from June 19-21. If you are even mildly curious about AI safety, this is a great way to start -- hands-on, practical and collaborative.

Please consider joining and share with students, builders, researchers and policy folks who might be interested.

[Apart Research @apartresearch]

AI safety research clusters in a few US/EU cities. The risks don't.

Global South AI Safety Hackathon, June 19-21. Join a hub across Latin America, Africa & Asia, or join online. $6,000 in prizes + Apart Fellowship invites for top teams.

https://t.co/ySwxkM8u84

[David Krueger 🦥 ⏸️ ⏹️ ⏪ @DavidSKrueger]

Signs that someone doesn't get it:
- They talk about "reskilling" or the "transition" to a post-AI economy
- They think we can stop AI from helping people make bioweapons
- They think we can defend against AI-enabled bioweapons.
- They talk about how bad it would be for the economy to regulate/slow/pause/stop AI
- They say "AI can't do X"
- They treat AI as a "US vs. China" issue
- They talk about AI as if it were a normal technology
- They think of AI companies as good faith actors
- They think AI is 100% hype
- They say "humans will always be in control" or "we could just unplug it" etc.
...

[Soumya Jain @wild_and_empty]

@tenobrus 1st jhana