Yes, that is technically correct. But SAT gives us a small, explicitly specified proof system and checker, even when the resulting “certificate” is huge. For AI-generated math that is formally checked, you are right that it has the same property.
The point here is that AI turns this from being an exceptional case into the normal mode of math: verification becomes cheap, while human understanding becomes the bottleneck.
We ran the largest open experiment on how frontier models do AI research.
100+ autonomous runs across 10+ models, sandboxed on 8xH200s for up to 8 days, iterating on the nanoGPT optimizer track.
Best runs closed 82% of the gap to a record built by dozens of humans over months.
Meet @amirbilu, our #Vim expert. Besides his infectious positivity and knack for handstands, Amir also loves designing awesome #dev T-shirts. 👕
🖍️What would you print on yours?
Hey, Bay Area #developers! Come join Tabnine and @Cloudflare this Wednesday for Rooftop Hack Night! It'll be a fun night of food, AI, and networking with a dynamite San Francisco view.
We hope to see you there!
https://t.co/zoelkKkON1
#SanFrancisco#event#AI
🎧 New episode alert! On @sdtimes "What the Dev" podcast, Tabnine’s @brandoncjung & DS Times David Rubinstein explore Apple's OpenAI partnership, transparency, and open source models. Listen here: https://t.co/KoXhZBiAGe
#AI#SDTimes#Developers#podcast
Tabnine is partnering with #TestMuConf 2024 by @lambdatesting! This virtual conference brings together #testers, #developers, thought leaders, and community builders from around the world.
🗓️Mark your calendars for August 21–23.
Register for free here: https://t.co/aNC5cvh5Yn
New week, new #LLM: Starting today, you can use @cohere's Command R model in Tabnine Chat to power code generation, #code explanations, documentation generation, AI-created tests, and more.
Get all the details: https://t.co/3gA0ZK3UV6
#coding#developers#AI
New week, new #LLM: Starting today, you can use @AnthropicAI’s Claude 3 Sonnet model in Tabnine #Chat to power #code generation, code explanations, documentation generation, AI-created tests, and more.
Get all the details: https://t.co/hqsi1UQg58
More than 50% of the reported reasoning abilities of LLMs might not be true reasoning.
How do we evaluate models trained on the entire internet? I.e., what novel questions can we ask of something that has seen all written knowledge? Below: new eval, results, code, and paper.
Functional benchmarks are a new way to do reasoning evals. Take a popular benchmark, e.g., MATH, and manually rewrite its reasoning into code, MATH(). Run the code to get a snapshot that asks for the same reasoning but not the same question. A reasoning gap exists if a model’s performance is different on snapshots. Big question: Are current SOTA models closer to gap 0 (proper reasoning) or gap 100 (lots of memorization)?
What we find: Gaps in the range of 58% to 80% in a bunch of SOTA models. Motivates us to build Gap 0 models.
We’re releasing the paper, code, and 3 snapshots of functional MATH() today.
arxiv draft: https://t.co/KtvWPc0R72
github repo: https://t.co/gzDVaxZ9yg
1/🧵
The Atlassian Team '24 is nearly here, and we’re headed back to Las Vegas! Swing by Booth #68 to connect with us and explore Tabnine's switchable models, personalization features, and our new Onboarding Agent. See You there!
#AtlassianTeam24#developers
https://t.co/cQ0YT04MNG
📢 Big new update: You can now instantly switch between the underlying LLMs that power Tabnine Chat based on your specific project or use case. 🪄
Get all the details 👉 https://t.co/KgE1aNnO2W
#LLMs#GenAI#Codinglife#AIcode#Developers
We will participate in #NVIDIA's next #meetup to discuss #LLMs for #developers. @yahave, our CTO, will be there along with influential field experts to discuss LLM use cases, valuable insights, and lessons learned.
Register here 👉🏽 https://t.co/CIdUZl5hBb
Exciting news: We’re partnering with @digitalocean to make it even easier for startups and SMBs to streamline the entire software development life cycle. Plus, get 25% off your monthly bill when you buy Tabnine Pro plans directly from your DigitalOcean account. 📷https://t.co/Y4kEoX3lV6 #Tabnine #DigitalOcean #DevOpsInnovation
"For all the mundane stuff, yeah, we’ve got your back. We can really turn you up to 11. You can be a better version of yourself as a developer."
CTO @yahave discussing with @DZoneInc how AI Coding Assistants are just another tool in your toolbelt.