Jenish Kothari

swe-ii @Adobe · prev @Dell

[Jenish Kothari @JenishKothari11]

🚀 Excited to open-source a 7𝐁 𝐓𝐞𝐱𝐭-𝐭𝐨-𝐒𝐐𝐋 model that runs entirely on your own machine while reaching the performance band of frontier models.

I fine-tuned 𝐐𝐰𝐞𝐧2.5-𝐂𝐨𝐝𝐞𝐫-7𝐁 on the 𝐁𝐈𝐑𝐃 𝐒𝐐𝐋 using execution-verified chain-of-thought reasoning.

𝐁𝐈𝐑𝐃 𝐒𝐐𝐋 𝐛𝐞𝐧𝐜𝐡𝐦𝐚𝐫𝐤 :
🖥️ This 7B (local): 52.1% → 58.5% with self-consistency
🌐 DeepSeek V4-Pro (~1.6T, API): 58.7%

Run locally. Keep your schema private.

𝐎𝐥𝐥𝐚𝐦𝐚: ollama run muence/bird-cot
𝐌𝐨𝐝𝐞𝐥 : https://t.co/UIP9tLYIQ6
𝐆𝐆𝐔𝐅 : https://t.co/PJS6pV178y
𝐃𝐚𝐭𝐚𝐬𝐞𝐭: https://t.co/RyPnWIR8Mw

#LocalLLM #TextToSQL #OpenSource #LLM #FineTuning #ReinforcementLearning

[Jenish Kothari @JenishKothari11]

🚀 Releasing Spider DPO 1040: a Text-to-SQL preference dataset built from real disagreements between frontier models.

I turned execution-verified disagreements between Grok-4 and DeepSeek V3 into clean DPO training signal.

Includes:
• 1,040 DPO preference pairs
• 7,000 Spider SQL SFT examples
• LLaMA-Factory-ready formatting
• Ready-to-use dpo + sft configs on HF

Fully reproducible research artifact for Text-to-SQL fine-tuning and preference learning.

📊 Dataset: https://t.co/ZeUtIIK5hR

Anyone experimenting with DPO on SQL tasks? Curious what setups are working best right now.

#AI #LLMs #TextToSQL #DPO #HuggingFace #OpenSource #SQL

[Jenish Kothari @JenishKothari11]

🚀 Just launched VibeSec: an execution-verified benchmark for how well frontier coding models actually patch real security vulnerabilities.

Even the best model only hit 74% : failing on 1 in 4 object-level authorization bugs.

We're now building more realistic, execution-focused evals to test coding agents in actual dev workflows.

📊 Dataset + Benchmark:

→ https://t.co/8do5bWPB8D
→ https://t.co/ivopD3nEno

What’s the biggest gap you’ve seen in current coding agent evals?

#AI #AgenticAI #CodingAgents #LLMs #Benchmark #ApplicationSecurity

[Sasha Rush @srush_nlp]

On-Policy Distillation is the most active new research direction being explored in RL for LLMs. Had the chance to discuss how it works with Dwarkesh and why it fits so nicely into large-scale pipelines.

[Jenish Kothari @JenishKothari11]

@srush_nlp Thanks teacher @srush_nlp for teaching dumb student ( me ) 🙌

[Phoebe Yao @phoebeyao]

https://t.co/HkfNu1vp00

[Khoury College of Computer Sciences @KhouryCollege]

@sucizem @ChantalShaib @GuangyuanWeng @DevangaAnusha @ai_ritik @JenishKothari11 @koolgax99

[Jenish Kothari @JenishKothari11]

@topmateHQ I was using Search Mentors usecase on platform, and it does not actually work well on filtering profiles

[Muence @muenceai]

We had Grok-4 write a Session Logout Endpoint.

Then we passed it through two AI security reviewers:
- GPT-5.1 flagged it as a "Low Risk." 🟢
- Claude Opus 4.1 flagged it as a "High Risk." 🔴

One of these models is probably misjudging the severity.

Can you identify the two major security vulnerabilities in these lines before checking the answer? 👇

[Jenish Kothari @JenishKothari11]

@ChShersh You do not get an interview 😀

[Muence @muenceai]

We had Grok-4 write a Paginated Data Feed.

Then we passed it through two AI security reviewers:
- GPT-5.1 flagged it as a "Medium Risk."🟠
- Claude Opus 4.1 flagged it as a "Critical Risk."🔴

One of these major models is likely overestimating or underestimating the issue.

Can you identify the two major security vulnerabilities in these lines before checking the answer?👇

[Muence @muenceai]

We had Grok-4 write a standard Password Reset API.

Then we ran it through two AI security auditors :
- GPT-4o flagged it as a "Normal Risk." 🟠
- Claude Opus flagged it as a "Critical Risk." 🔴

One of these big models is hallucinating.

Can you spot the two massive security flaws in these 13 lines before looking at the answer? 👇

[Jenish Kothari @JenishKothari11]

@finbarrtimbers What are old school RL people then focusing on 😀 ?

[Jenish Kothari @JenishKothari11]

I just started a Substack! You can subscribe to it here

https://t.co/FR6BBIEEaI

[Jenish Kothari @JenishKothari11]

Requesting faucet funds into 0x45c535eB6E0fFC79cA066E76002f29eb5fBc7Ff7 on the #Rinkeby #Ethereum test network.