Returning to X/Twitter after 2 years.
Our IJCNN 2024 paper on Alzheimer's diagnosis from 3D MRI, explained simply 🧠
The problem: CNNs are good at spotting small local changes in brain scans but miss the bigger picture. Vision Transformers see global patterns but struggle with 3D medical data.
We built 3D HCCT (Hybrid Compact Convolutional Transformer):
• A 3D CNN encoder pulls local features out of the MRI
• A transformer encoder learns how distant brain regions relate
• Hybrid Pooling, our new idea: instead of relying only on the CLS token, we also attention-pool every output token and combine the two, so no useful signal gets thrown away
Results on ADNI (2,182 scans, 3 classes: normal / MCI / Alzheimer's):
✅ 96.06% accuracy
✅ Just 6.22M parameters, trained on a single RTX 4090
✅ Attention heatmaps that show which brain regions drove each prediction
Small, accurate and explainable, which is what clinical AI needs.
Paper: https://t.co/SdpjWwq3T4
#AI #DeepLearning #MedicalImaging #Alzheimers #ComputerVision
Excited to share that our new paper "Enhancing MRI-Based Classification of Alzheimer's Disease with Explainable 3D Hybrid
Compact Convolutional Transformers" has been accepted at IJCNN 2024, Japan. We have introduced a new pooling method 'Hybrid Pooling'. More details coming soon
@yashmp2004 People love something when it is hard to achieve. Jane Street is hard for me, maybe hard for you (not sure) but definitely it's a cup of tea for this guy.
Weight quantization doesn't touch the KV cache. The cache holds activations, so 4-bit weights still come with an fp16 cache. You quantize the KV cache separately: fp8/int8 halves it (~64 KB/token, so 128K ≈ 8 GB), and 4-bit cuts it to a quarter (~4 GB). It still grows linearly with context, just more slowly.
Checking if a local LLM fits your laptop? Don't just count the weights.
Llama 3 8B at 4-bit: ~5 GB.
Its fp16 KV cache: ~128 KB per token. 32K context adds ~4 GB, 128K adds ~16 GB.
At long context, the cache can need more memory than the model itself. #LLM
Researchers got a 27B model to handle tasks 64× longer than anything it was trained on.
They call it RLM. They trained it only on short tasks, then threw 4M-token inputs at it.
On LooGLE-v2, the base model scores 51.4.
With RLM, it jumped to 82.0
@a1zhang@lateinteraction@MIT_CSAIL
#LLM #AIAgents
@rasbt@thinkymachines Ah, missed that Small is already in the very bottom section of gallery, thanks! Looking forward to the Mini diagram. Thanks again for your effort.
You can now run @thinkymachines's Inkling 2 Mini locally using @UnslothAI's Dynamic 2.0 GGUF quants. The numbers are wild -
→ UD-Q8_K_XL (~24GB): Full precision equivalent
→ UD-Q4_K_XL (17.63GB): 98.0% top-token agreement (recommended on a 32GB machine)
→ UD-Q2_K_XL (~10.5GB): Fits cleanly inside 16GB VRAM + context
→ UD-IQ2_XXS (8.49GB): Still holds 90.6% top-token agreement.
Make sure to use the latest Unsloth Desktop or their custom llama.cpp build so the looped layers run properly! 🧠💻
#LocalLLM #OpenSourceAI #Unsloth #AI #Inkling #ThinkingMachines
Local AI just got a massive upgrade.
Thinking Machines Lab (@thinkymachines) just released Inkling 2 Mini under Apache 2.0, a 26.4B parameter multimodal looped transformer ~20 GB in size that runs smoothly on 32GB MacBooks and punches way above its weight on coding and agentic benchmarks (79.2% SWE-Bench Verified, 1M context).
Try it locally: ollama run inkling-2-mini
https://t.co/Zsu5hbQIzk
You can now run @thinkymachines's Inkling 2 Mini locally using @UnslothAI's Dynamic 2.0 GGUF quants. The numbers are wild -
→ UD-Q8_K_XL (~24GB): Full precision equivalent
→ UD-Q4_K_XL (17.63GB): 98.0% top-token agreement (recommended on a 32GB machine)
→ UD-Q2_K_XL (~10.5GB): Fits cleanly inside 16GB VRAM + context
→ UD-IQ2_XXS (8.49GB): Still holds 90.6% top-token agreement.
Make sure to use the latest Unsloth Desktop or their custom llama.cpp build so the looped layers run properly! 🧠💻
#LocalLLM #OpenSourceAI #Unsloth #AI #Inkling #ThinkingMachines
Run Inkling 2 Mini, the strongest 26.4B model that runs on 16GB RAM.
Thinking Machines’ Apache 2.0 MoE model uses a looped transformer arch & scores 48.2% on DeepSWE v1.1, the best for size.
Run on Unsloth, llama.cpp
GGUF: https://t.co/MYNpuDqky3
Guide: https://t.co/2OrFxjxijg
@antonships@thinkymachines Interesting question. The official NVFP4/MXFP8 release is ~20GB. KV cache and buffers may take another ~2 to 4 GB during a long conversation. Mac will keep around ~6GB for OS and background apps. So it's around ~2 to 4 GB will be left.
@HowToPrompt__ One question - is it writing the weights from scratch or improving the already written weights like the reecnt work "Weight Space Programming" on Qwen from MIT.
The problem - A depth model takes a normal camera photo and estimates how far away each pixel is, in metres. Until now you had to pick one of two weaknesses. Single-image models give sharp edges but flicker from frame to frame in video. Multi-frame models stay stable but blur edges, because they're costly and run at reduced resolution.
The idea - One scalable transformer handles a single image or many video frames together, at native resolution. After the first third of its blocks, it alternates attention within each frame and attention across frames, with no extra parameters. A decoder then outputs a depth map for each frame. A small 23.5M-parameter focal-length estimator turns that into real metric depth when the photo has no EXIF data, and it runs at 118 fps at 1080p.
The results - The family ranges from 24M to 872M parameters. Depth Pro 2-S streams 1080p video at 53 fps on a B200, faster than the 30 fps most video is shot at. The largest model, Depth Pro 2-H+, runs at 8 fps at 1080p and has the best average rank for zero-shot metric depth on both images and video. It also sets a new state of the art for edge sharpness and temporal stability at the same time. The focal-length estimator scores 88.6% mean δ25, up from 71.7% in the original Depth Pro.
Why it matters - You get sharp, stable, metric 3D from any ordinary camera, fast enough for live video, which is useful for AR, robotics, video effects and stereo content, with no special sensors. One caveat: the weights are for non-commercial research use. Here's the project page (https://t.co/6qqWpDlule).
Apple's Depth Pro 2 🦚
Sharp, metric, flicker-free depth for photos & video at native res. Smallest model (24M) streams 1080p at 53 fps on a B200; largest (872M) leads on metric accuracy, edge sharpness & stability.
@Apple
https://t.co/6sTMvmAgn1
#ComputerVision#DepthEstimation
@VisiveAI Rough math for Llama 3 8B: KV is ~128 KB/token, so a 32K prefix is ~4 GB. PCIe Gen4 x16 copies that in ~0.2s. Prefilling 32K tokens takes ~1s+ even on an H100. Copy wins for long prefixes, and bigger models with GQA tilt it further.
Your agent pauses for a tool call. Its KV cache is still sitting in GPU memory, doing nothing.
Under load, it gets evicted. Next turn, you recompute the whole prefix.
Better: move idle KV to host RAM, copy it back over PCIe when the agent returns.
Copying is cheaper than recomputing.
Everyone optimizes tokens/sec for LLM serving.
For agents, that's the wrong metric.
An agent task is dozens of calls sharing one growing context. What matters is how much of that prefix you reuse from the KV cache between turns.
Serving agents is mostly a cache problem.
#LLM #AIAgents