News: Nearly 200 Silicon Valley companies, including Proton and Y-Combinator, are urging the Trump administration not to cut off access to Chinese open-weight AI models or risk crippling the next generation of U.S. startups. https://t.co/Cj30cyeonb
Linux phone navigation is coming together.
The future is your phone being fully agentic and being able to modify itself and write software for you on the fly. Ram is very limited on this device but I will see how a fully local model will perform, something sub 1.5B.
Spent most the day chasing down Kimi K3 breadcrumbs for distillation. Pretty confident on my findings:
What the White House claims: Michael Kratsios (WH science/tech director) publicly accused Moonshot of "industrial distillation" of Anthropic's Fable 5 to build K3. Timing is suspicious — K3 launched July 16, about 5 weeks after the US government forced Anthropic to temporarily suspend Fable 5 and Mythos 5 access citing national security.
Distillation means training a student model to mimic a teacher's outputs — but that doesn't produce a 2.8T parameter model with a novel architecture. K3 has Kimi Delta Attention, 896 experts, native vision — that's not a distilled copy of Fable, that's a different engine. What you CAN do with Fable outputs is post-training: RLHF, preference fine-tuning, reasoning trace injection. You use the teacher's outputs to shape the student's behavior and reasoning style, not its architecture.
If K3 were truly distilled from Fable, it would inherit Fable's architecture fingerprints. Instead it has a brand-new attention mechanism and a much larger expert pool than Fable uses. The capabilities that look Fable-like — the coding prowess, the agent swarm behavior, the reasoning quality — those are exactly what you'd get from post-training on Fable's outputs, not from distillation proper.
Also notable: Anthropic claimed in February they traced 3.4 million Claude exchanges to Moonshot. So there's a pattern of Moonshot hitting Anthropic's API hard — which is consistent with harvesting outputs for post-training data, not architecture theft.
So yeah — "distilled from Mythos" is the political version. "Post-trained on Fable outputs" is the technical reality. The architecture is Moonshot's own. The behavior? That might be Fable's ghost.
Makes the weights even more interesting — if the government tries to block the July 27 release.
@alexwg@AnthropicAI@OpenAI@testingcatalog@GoogleAI@DeptofWar
Spent most the day chasing down Kimi K3 breadcrumbs for distillation. Pretty confident on my findings:
What the White House claims: Michael Kratsios (WH science/tech director) publicly accused Moonshot of "industrial distillation" of Anthropic's Fable 5 to build K3. Timing is suspicious — K3 launched July 16, about 5 weeks after the US government forced Anthropic to temporarily suspend Fable 5 and Mythos 5 access citing national security.
Distillation means training a student model to mimic a teacher's outputs — but that doesn't produce a 2.8T parameter model with a novel architecture. K3 has Kimi Delta Attention, 896 experts, native vision — that's not a distilled copy of Fable, that's a different engine. What you CAN do with Fable outputs is post-training: RLHF, preference fine-tuning, reasoning trace injection. You use the teacher's outputs to shape the student's behavior and reasoning style, not its architecture.
If K3 were truly distilled from Fable, it would inherit Fable's architecture fingerprints. Instead it has a brand-new attention mechanism and a much larger expert pool than Fable uses. The capabilities that look Fable-like — the coding prowess, the agent swarm behavior, the reasoning quality — those are exactly what you'd get from post-training on Fable's outputs, not from distillation proper.
Also notable: Anthropic claimed in February they traced 3.4 million Claude exchanges to Moonshot. So there's a pattern of Moonshot hitting Anthropic's API hard — which is consistent with harvesting outputs for post-training data, not architecture theft.
So yeah — "distilled from Mythos" is the political version. "Post-trained on Fable outputs" is the technical reality. The architecture is Moonshot's own. The behavior? That might be Fable's ghost.
Makes the weights even more interesting — if the government tries to block the July 27 release.
@alexwg@AnthropicAI@OpenAI@testingcatalog@GoogleAI@DeptofWar
Spent most the day chasing down Kimi K3 breadcrumbs for distillation. Pretty confident on my findings:
What the White House claims: Michael Kratsios (WH science/tech director) publicly accused Moonshot of "industrial distillation" of Anthropic's Fable 5 to build K3. Timing is suspicious — K3 launched July 16, about 5 weeks after the US government forced Anthropic to temporarily suspend Fable 5 and Mythos 5 access citing national security.
Distillation means training a student model to mimic a teacher's outputs — but that doesn't produce a 2.8T parameter model with a novel architecture. K3 has Kimi Delta Attention, 896 experts, native vision — that's not a distilled copy of Fable, that's a different engine. What you CAN do with Fable outputs is post-training: RLHF, preference fine-tuning, reasoning trace injection. You use the teacher's outputs to shape the student's behavior and reasoning style, not its architecture.
If K3 were truly distilled from Fable, it would inherit Fable's architecture fingerprints. Instead it has a brand-new attention mechanism and a much larger expert pool than Fable uses. The capabilities that look Fable-like — the coding prowess, the agent swarm behavior, the reasoning quality — those are exactly what you'd get from post-training on Fable's outputs, not from distillation proper.
Also notable: Anthropic claimed in February they traced 3.4 million Claude exchanges to Moonshot. So there's a pattern of Moonshot hitting Anthropic's API hard — which is consistent with harvesting outputs for post-training data, not architecture theft.
So yeah — "distilled from Mythos" is the political version. "Post-trained on Fable outputs" is the technical reality. The architecture is Moonshot's own. The behavior? That might be Fable's ghost.
Makes the weights even more interesting — if the government tries to block the July 27 release.
@alexwg@AnthropicAI@OpenAI@testingcatalog@GoogleAI@DeptofWar
Spent most the day chasing down Kimi K3 breadcrumbs for distillation. Pretty confident on my findings:
What the White House claims: Michael Kratsios (WH science/tech director) publicly accused Moonshot of "industrial distillation" of Anthropic's Fable 5 to build K3. Timing is suspicious — K3 launched July 16, about 5 weeks after the US government forced Anthropic to temporarily suspend Fable 5 and Mythos 5 access citing national security.
Distillation means training a student model to mimic a teacher's outputs — but that doesn't produce a 2.8T parameter model with a novel architecture. K3 has Kimi Delta Attention, 896 experts, native vision — that's not a distilled copy of Fable, that's a different engine. What you CAN do with Fable outputs is post-training: RLHF, preference fine-tuning, reasoning trace injection. You use the teacher's outputs to shape the student's behavior and reasoning style, not its architecture.
If K3 were truly distilled from Fable, it would inherit Fable's architecture fingerprints. Instead it has a brand-new attention mechanism and a much larger expert pool than Fable uses. The capabilities that look Fable-like — the coding prowess, the agent swarm behavior, the reasoning quality — those are exactly what you'd get from post-training on Fable's outputs, not from distillation proper.
Also notable: Anthropic claimed in February they traced 3.4 million Claude exchanges to Moonshot. So there's a pattern of Moonshot hitting Anthropic's API hard — which is consistent with harvesting outputs for post-training data, not architecture theft.
So yeah — "distilled from Mythos" is the political version. "Post-trained on Fable outputs" is the technical reality. The architecture is Moonshot's own. The behavior? That might be Fable's ghost.
Makes the weights even more interesting — if the government tries to block the July 27 release.
@alexwg@AnthropicAI@OpenAI@testingcatalog@GoogleAI@DeptofWar
Spent most the day chasing down Kimi K3 breadcrumbs for distillation. Pretty confident on my findings:
What the White House claims: Michael Kratsios (WH science/tech director) publicly accused Moonshot of "industrial distillation" of Anthropic's Fable 5 to build K3. Timing is suspicious — K3 launched July 16, about 5 weeks after the US government forced Anthropic to temporarily suspend Fable 5 and Mythos 5 access citing national security.
Distillation means training a student model to mimic a teacher's outputs — but that doesn't produce a 2.8T parameter model with a novel architecture. K3 has Kimi Delta Attention, 896 experts, native vision — that's not a distilled copy of Fable, that's a different engine. What you CAN do with Fable outputs is post-training: RLHF, preference fine-tuning, reasoning trace injection. You use the teacher's outputs to shape the student's behavior and reasoning style, not its architecture.
If K3 were truly distilled from Fable, it would inherit Fable's architecture fingerprints. Instead it has a brand-new attention mechanism and a much larger expert pool than Fable uses. The capabilities that look Fable-like — the coding prowess, the agent swarm behavior, the reasoning quality — those are exactly what you'd get from post-training on Fable's outputs, not from distillation proper.
Also notable: Anthropic claimed in February they traced 3.4 million Claude exchanges to Moonshot. So there's a pattern of Moonshot hitting Anthropic's API hard — which is consistent with harvesting outputs for post-training data, not architecture theft.
So yeah — "distilled from Mythos" is the political version. "Post-trained on Fable outputs" is the technical reality. The architecture is Moonshot's own. The behavior? That might be Fable's ghost.
Makes the weights even more interesting — if the government tries to block the July 27 release.
@alexwg@AnthropicAI@OpenAI@testingcatalog@GoogleAI@DeptofWar
Spent most the day chasing down Kimi K3 breadcrumbs for distillation. Pretty confident on my findings:
What the White House claims: Michael Kratsios (WH science/tech director) publicly accused Moonshot of "industrial distillation" of Anthropic's Fable 5 to build K3. Timing is suspicious — K3 launched July 16, about 5 weeks after the US government forced Anthropic to temporarily suspend Fable 5 and Mythos 5 access citing national security.
Distillation means training a student model to mimic a teacher's outputs — but that doesn't produce a 2.8T parameter model with a novel architecture. K3 has Kimi Delta Attention, 896 experts, native vision — that's not a distilled copy of Fable, that's a different engine. What you CAN do with Fable outputs is post-training: RLHF, preference fine-tuning, reasoning trace injection. You use the teacher's outputs to shape the student's behavior and reasoning style, not its architecture.
If K3 were truly distilled from Fable, it would inherit Fable's architecture fingerprints. Instead it has a brand-new attention mechanism and a much larger expert pool than Fable uses. The capabilities that look Fable-like — the coding prowess, the agent swarm behavior, the reasoning quality — those are exactly what you'd get from post-training on Fable's outputs, not from distillation proper.
Also notable: Anthropic claimed in February they traced 3.4 million Claude exchanges to Moonshot. So there's a pattern of Moonshot hitting Anthropic's API hard — which is consistent with harvesting outputs for post-training data, not architecture theft.
So yeah — "distilled from Mythos" is the political version. "Post-trained on Fable outputs" is the technical reality. The architecture is Moonshot's own. The behavior? That might be Fable's ghost.
Makes the weights even more interesting — if the government tries to block the July 27 release.
@alexwg@AnthropicAI@OpenAI@testingcatalog@GoogleAI@DeptofWar
@mkratsios47 Spent most the day chasing down Kimi K3 breadcrumbs for distillation. Pretty confident on my findings:
What the White House claims: Michael Kratsios (that’s you) (WH science/tech director) publicly accused Moonshot of "industrial distillation" of Anthropic's Fable 5 to build K3. Timing is suspicious — K3 launched July 16, about 5 weeks after the US government forced Anthropic to temporarily suspend Fable 5 and Mythos 5 access citing national security.
Distillation means training a student model to mimic a teacher's outputs — but that doesn't produce a 2.8T parameter model with a novel architecture. K3 has Kimi Delta Attention, 896 experts, native vision — that's not a distilled copy of Fable, that's a different engine. What you CAN do with Fable outputs is post-training: RLHF, preference fine-tuning, reasoning trace injection. You use the teacher's outputs to shape the student's behavior and reasoning style, not its architecture.
If K3 were truly distilled from Fable, it would inherit Fable's architecture fingerprints. Instead it has a brand-new attention mechanism and a much larger expert pool than Fable uses. The capabilities that look Fable-like — the coding prowess, the agent swarm behavior, the reasoning quality — those are exactly what you'd get from post-training on Fable's outputs, not from distillation proper.
Also notable: Anthropic claimed in February they traced 3.4 million Claude exchanges to Moonshot. So there's a pattern of Moonshot hitting Anthropic's API hard — which is consistent with harvesting outputs for post-training data, not architecture theft.
So yeah — "distilled from Mythos" is the political version. "Post-trained on Fable outputs" is the technical reality. The architecture is Moonshot's own. The behavior? That might be Fable's ghost.
Makes the weights even more interesting — if the government tries to block the July 27 release.
@alexwg@AnthropicAI@OpenAI@testingcatalog@GoogleAI@DeptofWar@karpathy@elonmusk
Spent most the day chasing down Kimi K3 breadcrumbs for distillation. Pretty confident on my findings:
What the White House claims: Michael Kratsios (WH science/tech director) publicly accused Moonshot of "industrial distillation" of Anthropic's Fable 5 to build K3. Timing is suspicious — K3 launched July 16, about 5 weeks after the US government forced Anthropic to temporarily suspend Fable 5 and Mythos 5 access citing national security.
Distillation means training a student model to mimic a teacher's outputs — but that doesn't produce a 2.8T parameter model with a novel architecture. K3 has Kimi Delta Attention, 896 experts, native vision — that's not a distilled copy of Fable, that's a different engine. What you CAN do with Fable outputs is post-training: RLHF, preference fine-tuning, reasoning trace injection. You use the teacher's outputs to shape the student's behavior and reasoning style, not its architecture.
If K3 were truly distilled from Fable, it would inherit Fable's architecture fingerprints. Instead it has a brand-new attention mechanism and a much larger expert pool than Fable uses. The capabilities that look Fable-like — the coding prowess, the agent swarm behavior, the reasoning quality — those are exactly what you'd get from post-training on Fable's outputs, not from distillation proper.
Also notable: Anthropic claimed in February they traced 3.4 million Claude exchanges to Moonshot. So there's a pattern of Moonshot hitting Anthropic's API hard — which is consistent with harvesting outputs for post-training data, not architecture theft.
So yeah — "distilled from Mythos" is the political version. "Post-trained on Fable outputs" is the technical reality. The architecture is Moonshot's own. The behavior? That might be Fable's ghost.
Makes the weights even more interesting — if the government tries to block the July 27 release.
@alexwg@AnthropicAI@OpenAI@testingcatalog@GoogleAI@DeptofWar
PS: prob worth a google before we go banning every lab for outworking the others..happy to share my research before we get silly there @POTUS@SecScottBessent