Abhishek Upperwal

Building Foundation Models • Founder at @soketlabs • @iiscbangalore • HPC ♥️ AI

[Abhishek Upperwal @upperwal]

Go for the hardest problem. Love it. Your team is definitely one of the best examples that frontier AI work can happen out of India

[Manish Gupta @ManishGuptaMG1]

Sharing my message that there is no shortage of AI talent in India. As our team has shown, you can do world class AI research from India.

[Abhishek Upperwal @upperwal]

Many youngsters are enthusiastically finding vulnerabilities in govt websites.

@IndianCERT should use this opportunity to launch a bug bounty program to incentivise and encourage them. Will build trust and also safeguard the Indian infra. Wont find a better opportunity than this.

[Abhishek Upperwal @upperwal]

@ThreatT0Society @soketlabs Thats a feature, not a bug. Will report it in the paper

[Abhishek Upperwal @upperwal]

Here’s an early signal to what I said

For almost an year now, we at @soketlabs have been working on curating a frontier scale pretraining data corpus along with finding the best architecture that fits the diversity of languages along with being compute optimal for both training and inference.

Sharing one of the many successes we have encountered. Our current version of the arch (yeah, its not a clone of deepseek or any other known arch) is at least 30% compute optimal to Deepseek’s sparse-MoE. These are just initial results and we hope to find a lot more. Shows that we have a lot more to learn about these architectures.

Also excited about the pretraining data we have curated but more on that later

Research efforts take time but they also yield exponential outcomes and thats what most people in India should be building towards

[Abhishek Upperwal @upperwal]

India will build one in the next 2 years

[Abhishek Upperwal @upperwal]

@Thomas_Tao_1 @soketlabs I agree. This was a controlled experiment and early results. More ablations are in the pipeline to look for the impact due to data quality.

[Abhishek Upperwal @upperwal]

https://t.co/ReQsSGLoGC

[Abhishek Upperwal @upperwal]

Here’s an early signal to what I said

For almost an year now, we at @soketlabs have been working on curating a frontier scale pretraining data corpus along with finding the best architecture that fits the diversity of languages along with being compute optimal for both training and inference.

Sharing one of the many successes we have encountered. Our current version of the arch (yeah, its not a clone of deepseek or any other known arch) is at least 30% compute optimal to Deepseek’s sparse-MoE. These are just initial results and we hope to find a lot more. Shows that we have a lot more to learn about these architectures.

Also excited about the pretraining data we have curated but more on that later

Research efforts take time but they also yield exponential outcomes and thats what most people in India should be building towards

[Abhishek Upperwal @upperwal]

India will build one in the next 2 years

[Shveta @TrustScore_1]

India has zero chance of building a frontier AI lab this decade.

[Abhishek Upperwal @upperwal]

@ni5arga Dude! Are you seriously 19? You should come and work with us.

[Abhishek Upperwal @upperwal]

@curious1yash Frontier AI is a non-trivial problem and research needs patience and time

[Abhishek Upperwal @upperwal]

@TrustScore_1 Many really smart people in multiple orgs are working towards it. Should be achievable

[Abhishek Upperwal @upperwal]

We at Soket AI are hiring for HPC Infra Engineer towards our foundation model efforts. Apply and do share within your network.

#HiringAlert #FoundationModel #AI #LLM #GPU

https://t.co/WMWz9o3Las

[Abhishek Upperwal @upperwal]

@w2sgarnav Do they even have capacity? They have very strict quota especially for free credits

[Abhishek Upperwal @upperwal]

@s_ketharaman Because their business model is obsolete in the age of AI

[Abhishek Upperwal @upperwal]

I hope Indian IT service companies now realise they need to work and invest in Indian startups.

[Chandra R. Srikanth @chandrarsrikant]

Anthropic partners Blackstone, Goldman Sachs to launch $1.5 billion AI Services venture

https://t.co/vPiMghY0C1

[Abhishek Upperwal @upperwal]

@loneyai Then we will be Anthropic one day and start our own service firm :)

[Abhishek Upperwal @upperwal]

@varchasvee_ Lol

[Abhishek Upperwal @upperwal]

Precisely! Very soon most of their work will be ingested in the model layer and they wont have much to offer

People also believe that AI will bring new demand for the service sector. That might be true in the immediate future because AI needs human intervention but AI will get a lot better and replace the AI service sector itself. Services is not a viable business model anymore. IP and innovation is the only way forward. Invest in R&D.

[Abhishek Upperwal @upperwal]

So we are moving toward chinese closed source models now.

Soon the argument that India should just “leverage open source” instead of building its own models will end up where it belongs - in the graveyard.

[𝗭𝗲𝗻 𝗠𝗮𝗴𝗻𝗲𝘁𝘀 @ZenMagnets]

Qwen3.6 397b will not be open source. This pretty much seals it. A real tragedy.

[Abhishek Upperwal @upperwal]

@dhruv2038 And govt relations. I dont think they will bring their r&d to India any time soon

[Abhishek Upperwal @upperwal]

Thats right. They are treating frontier AI like traditional software.

Many of these large IT services orgs have partnerships with OpenAI and Anthropic hoping it would save their business model. Now they are realising that these labs are eating their business one segment at a time and yet they continue to wait and watch

[Abhishek Upperwal @upperwal]

@Hardik_Meisheri Thats so true. Have spoken to many of these leaders. They are excited but no one wants to wander too far into the unknown. One mistake and they will be blamed for breaking what already works