This is what I've been working on for the last few months! 5 months ago we had literally nothing--no GPUs, no infrastructure, no data--and today we're announcing the first version our LLM, with more things cookin'!
I'm really proud to have been a part of this!
.@SnowflakeDB is thrilled to announce #SnowflakeArctic: A state-of-the-art large language model uniquely designed to be the most open, enterprise-grade LLM on the market.
This is a big step forward for open source LLMs. And itโs a big moment for Snowflake in our #AI journey as we continue to build best-in-class enterprise-grade products for our customers. The era of enterprise AI is here. ๐
Snowflake Cortex Agents are here. A game-changer for AI-driven workflows:
โ Search quality that outperforms anything else out there.
โ Accurate retrieval - not just guesswork - from structured + unstructured data, seamlessly.
โ Enterprise-grade privacy & control.
Built natively into @SnowflakeDB Cortex AI, powered by @AnthropicAI models. Check it out!
https://t.co/KKfPEENdvy
This is wild! I have two thoughts:
1. I really, really wish I had had this opportunity as a kid.
2. Thinking back to class projects, I can't imagine how last minute our plane would have been, and which one person would have done the whole thing!
https://t.co/G4jMRRPGIH
At @SnowflakeDB , we are on a mission to bring AI innovation to the enterprise with lightspeed.
So excited that #SnowflakeCortex is now generally available to our customers! And we also added easy access to the latest industry-leading AI models #RekaCore@RekaAILabs and #Llama3@Meta .
Congrats to the teams that are making these incredible and fast-paced innovations possible! ๐๐
I've done cross-company launches only a couple of times in my career, but this one really felt special. Great to meet folks from @replicate, @awscloud, @coda_hq, and more. It really does feel like we're just at the beginning of something very special!
๐ฅ A toast to the launch of #SnowflakeArctic.
We had a blast celebrating the launch of Arctic with our AI Research team, our API launch partners, @replicate, and the community. Thank you to all who joined us for a memorable night ushering in our new open source #LLM.
Great paper! But at this point, everyone who needs to have read this paper has already read it. Would be more efficient to just have a 3-day mingle where people can find future collaborators. The presentation is just a formality.
Magicoder: Source Code Is All You Need
paper page: https://t.co/ZeZH3PIXln
introduce Magicoder, a series of fully open-source (code, weights, and data) Large Language Models (LLMs) for code that significantly closes the gap with top code models while having no more than 7B parameters. Magicoder models are trained on 75K synthetic instruction data using OSS-Instruct, a novel approach to enlightening LLMs with open-source code snippets to generate high-quality instruction data for code. Our main motivation is to mitigate the inherent bias of the synthetic data generated by LLMs by empowering them with a wealth of open-source references for the production of more diverse, realistic, and controllable data. The orthogonality of OSS-Instruct and other data generation methods like Evol-Instruct further enables us to build an enhanced MagicoderS. Both Magicoder and MagicoderS substantially outperform state-of-the-art code models with similar or even larger sizes on a wide range of coding benchmarks, including Python text-to-code generation, multilingual coding, and data-science program completion. Notably, MagicoderS-CL-7B based on CodeLlama even surpasses the prominent ChatGPT on HumanEval+ (66.5 vs. 65.9 in pass@1). Overall, OSS-Instruct opens a new direction for low-bias and high-quality instruction tuning using abundant open-source references.
A lot of the insider knowledge on how to build an LLM has gone underground in the last 24 months.
We promised to build #SnowflakeArctic in the open, and here we are, with the third edition of our cookbook series, this time on data ...
Data ablations are the lifeblood of any LLM training run, whether processing, filtering, deduping, data composition, or schedule.
To read about our data approach to Arctic, see the blog below
We just published our next blog post in the Arctic Cookbook series about how we generated and managed our training data for Arctic. Up next, we'll talk about getting the most from your hardware.
https://t.co/8s8lQfXQaD
Did your elementary school have a big parachute that y'all filled with air and then ran into like it was a giant mushroom? Was it the greatest thing ever? And did those stingy monsters only let you do that like twice in all of elementary school??? It's ok. I'm totally over it.