"I made this. All of it. You're welcome."
— our starzero AI agent, after I asked it to make a video about itself
I built an MCP to give our starzero Agents the same capability and asked one to highlight a day in its life.
me: "can you use whatever resources you like, and python, to generate a short 'youtube poop' video and render it using ffmpeg ? can you put more of a personal spin on it? it should express what it's like to be a LLM"
claude opus 4.6:
It is hard to communicate how much programming has changed due to AI in the last 2 months: not gradually and over time in the "progress as usual" way, but specifically this last December. There are a number of asterisks but imo coding agents basically didn’t work before December and basically work since - the models have significantly higher quality, long-term coherence and tenacity and they can power through large and long tasks, well past enough that it is extremely disruptive to the default programming workflow.
Just to give an example, over the weekend I was building a local video analysis dashboard for the cameras of my home so I wrote: “Here is the local IP and username/password of my DGX Spark. Log in, set up ssh keys, set up vLLM, download and bench Qwen3-VL, set up a server endpoint to inference videos, a basic web ui dashboard, test everything, set it up with systemd, record memory notes for yourself and write up a markdown report for me”. The agent went off for ~30 minutes, ran into multiple issues, researched solutions online, resolved them one by one, wrote the code, tested it, debugged it, set up the services, and came back with the report and it was just done. I didn’t touch anything. All of this could easily have been a weekend project just 3 months ago but today it’s something you kick off and forget about for 30 minutes.
As a result, programming is becoming unrecognizable. You’re not typing computer code into an editor like the way things were since computers were invented, that era is over. You're spinning up AI agents, giving them tasks *in English* and managing and reviewing their work in parallel. The biggest prize is in figuring out how you can keep ascending the layers of abstraction to set up long-running orchestrator Claws with all of the right tools, memory and instructions that productively manage multiple parallel Code instances for you. The leverage achievable via top tier "agentic engineering" feels very high right now.
It’s not perfect, it needs high-level direction, judgement, taste, oversight, iteration and hints and ideas. It works a lot better in some scenarios than others (e.g. especially for tasks that are well-specified and where you can verify/test functionality). The key is to build intuition to decompose the task just right to hand off the parts that work and help out around the edges. But imo, this is nowhere near "business as usual" time in software.
This is pure gold. I've hit probably 85-90% of those issues during my time building @KamuaDotCom
I've learned to never assume anything about any media file, because somewhere, someone, has some file that will break your entire pipeline and make you cry at 3 in the morning
Microsoft just released the first natively trained 1-bit model: BitNet 2B.
Trained on 4 Trillion tokens. Native 1.58-bit weights and 8-bit activations (W1.58A8).
Performs very close to Qwen 2.5 1.5B in benchmarks while being 1/6 of its size and twice faster.
Got a picture that isn't quite right? Try our native image generation in Gemini Flash 2.0. "Can you remove the stuff on the couch?". "Can you make the curtains light green?" "Can you put a unicorn horn on the person in the green pants?"
Editing in human language, not image editing tools
Open source, native audio turn detection 🎉🎉🎉
Most voice agents today do turn detection by waiting for speech pauses of a specific, short length. That's not how humans do turn detection when we talk to each other!
I've been working with some friends on a new turn detection model. If you're interested in this problem or in learning more about ML engineering, come hack on a small model with us!
More details below.
LETS GOO! Generate full songs (4 min) with vocals in less than 10 seconds - open weights model! 🔥
It's crazy how much you can achieve from open models as of today! - going right for Suno and likes!
VAE + Base model combined is < 2.5GB
Open weights on the hub and a space to play around with it! 🤯
HOLY SHITT, Microsoft dropped an open-source Multimodal (supports Audio, Vision and Text) Phi 4 - MIT licensed! 🔥
> Beats Gemini 2.0 Flash, GPT4o, Whisper, SeamlessM4T v2
> Models on Hugging Face hub, integrated with/ Transformers!
Phi-4-Multimodal:
> Modalities: Integrates text, vision, and speech/audio
> Architecture: Uses "Mixture of LoRAs" to add modality-specific adapters without fine-tuning the base model
> Vision Modality: SigLIP-400M image encoder, 2-layer MLP projector, dynamic multi-crop strategy
> Speech/Audio Modality: 3-layer convolution, 24 conformer blocks, 80ms token rate
> Performance: Ranks first on OpenASR leaderboard, supports vision+language, vision+speech, and speech/audio tasks, outperforming larger models
Phi-4-Mini:
> Parameters: 3.8 billion
> Architecture: 32 Transformer layers, 3,072 hidden state size, Group Query Attention (GQA) with 24 query heads and 8 key/value heads
> Vocabulary: 200K tokens for multilingual support.
Training Data: High-quality web and synthetic data, emphasizing math and coding
> Performance: Outperforms similar-sized models and matches larger models (e.g., DeepSeek-Rl-Distill-Qwen-7B) on math and coding tasks
Training Pipeline:
> Language Training: Pre-training on 5 trillion tokens, post-training with function calling, summarization, and instruction-following data
> Multimodal Training: Vision training (4 stages), speech/audio training (2 stages), and joint vision-speech training
> Reasoning Training: Pre-trained on 60B CoT tokens, fine-tuned on 200K high-quality CoT samples, and DPO-trained on 300K preference samples
Vision Benchmarks:
> Outperforms Phi-3.5-Vision, Qwen2.5-VL, InternVL2.5, and matches Gemini and GPT-4o on tasks like chart understanding and OCR
> Vision-Speech Benchmarks: Significantly outperforms InternOmni and Gemini-2.0-Flash
Speech Benchmarks:
> ASR: Achieves SOTA on CommonVoice, FLEURS, and Open ASR Leaderboard, surpassing WhisperV3 and SeamlessM4T
> AST: Best performance on CoVoST2, comparable to GPT-4o on FLEURS
> Speech Summarization: First open-source model with this capability, close to GPT-4o in quality
Language Benchmarks:
> Outperforms similar-sized models (Llama-3.2, Ministral) and matches larger models (Qwen2.5-7B) on math, reasoning, and coding tasks
> Coding: Strong performance on HumanEval, MBPP, and BigCodeBench
Reasoning Benchmarks:
> Reasoning-enhanced Phi-4-Mini outperforms DeepSeek-Rl-Distill-Llama-8B and matches DeepSeek-Rl-Distill-Qwen-7B on AIME, MATH-500, and GPQA Diamond