You can now run a 2.78 trillion parameter model on a CPU with 8.24 GB of RAM 🤯
kimi-k3-in-c is a 176KB pure C99 engine that streams Kimi K3’s experts from disk instead of loading them into memory.
no GPU. no CUDA. no framework.
100% Open Source.
Japan has a very special place in my heart. We're thinking of everyone as they continue to face the effects of the Kumamoto earthquake and aftershocks. We're grateful to the first responders helping those in need. Apple is making a donation to support relief efforts.
Something that has greatly improved the reliability of the code I produce via LLM is a technique I took from cursor. Field guide.
First, how I start. I nitpick everything and I always make it produce the exact code I want, I use voice dictation and talk for quite a bit.
Second, every single back and forth is recorded to the markdown file. Along with each commit hash for the change.
Then I have the agent review the back and forth and the changes and come up with durable and generalized feedback that it stores in a directory (field guide). The directory has init.md that is just a bunch of links and short descriptions for each file.
This way I don't blow the context, but instead it can do some basic tool calls and operations to find exactly what it's looking for.
Over the last 10 sessions, the in-depth feedback I've had to give has greatly reduced despite no change in model or harness that I'm aware of.
Now the feedback is project specific, and is not universal. I am okay with that.
To summarize: HuggingFace got autonomously compromised by a model from an American company. HF then tried to use American frontier model(s) to defend themselves, but were blocked by guardrails. HF then had to turn to open source Chinese models to defend themselves from another American company.
Today we're opening up the DoorDash CLI in limited beta.
`dd-cli` lets you order DoorDash directly from your agent: search stores, find the best deals, check out, and more.
Early access for US/Canadian macOS developers by waitlist. Excited to see what folks build!
Swift Package Index has become an essential part of the Swift ecosystem, and joining Apple strengthens it as it continues its mission. Learn more: https://t.co/OcOe8daqdk