🎉 OASIS is accepted to NeurIPS 2026!
🔗A follow-up to Kimi Attention Residuals @Kimi_Moonshot
⚠️ We find that the extra depth-wise Softmax in Attention Residuals can amplify attention sinks, activation outliers, and low-bit quantization errors.
💡 OASIS adds explicit null routes to both token and depth routing, then feeds token-level null signals back into depth routing.
📈 Across multiple models:
↓95.9% kurtosis / ↓82.0% W8A8 PPL / ↑42.1% W4A4 GSM8K
📚 On 12K context, OASIS also substantially recovers the performance drop of AttnResidual on RULER and LongBench.
✨ More robust Attention Residuals for low-bit inference and long context.
Thanks to all the collaborators across 8 institutes! @robinluo1997@HHarryD@Michael_Huang_W@NorthwesternU@illinoistech@RutgersU@UMich@UCLA@UCSD@TAMU@northwesterncs@iitcsdept@RutgersCS
Paper Link: https://t.co/4WLtfVv3K2
Project Website: https://t.co/S3m8Y4PcV3
Introducing Arrow 2
Our latest and most advanced models for generating precise, editable vector graphics.
Higher quality. Faster outputs. Available now in App and API.
Introducing QuiverAI MCP
A hosted MCP server bringing structured vector generation directly into agent workflows.
Generate SVGs from prompts, vectorize raster images, browse previous creations, track tasks, and retrieve completed SVG content from tools like Cursor and Codex.
Available now in beta: https://t.co/x2hznHOdfw
One thing AI has struggled with until now: technical drawings.
@QuiverAI unlocks generating and vectorizing CAD-like visuals into clean SVG.
Powered by a shift toward visual programming.
Introducing @QuiverAI, a new AI lab and product company focused on frontier vector design.
We’ve raised an $8.3M seed round led by @a16z, with support from amazing angels and investors.
Our first model, Arrow-1.0, generates SVGs from images and text. It’s available now in public beta at https://t.co/zjAnKlI8pp