Shahir

On-device ML engineer and Researcher: fine-tuning, ternary models, and Apple Neural Engine inference. Apple Platforms Research @C_engines

[Shahir @TslShahir]

@DeshpandeV13 , Hriddhi, Alex, and I had 24 hours to tackle a huge universal challenge during the @the_builderclub @GroqInc @Sahha_Health and Eucalyptus healthtech hackathon- How to sustain motivation during a weight loss journey, and..... 🥁🥁🥁we are one of the winners :D 🥳🥳🥳🥳
read on...(1/6)

[Shahir @TslShahir]

Four runs on a 203M ternary model, following Induction Labs' intrinsic-discovery loop — on text, then on a tool surface. https://t.co/PALx7tBw3H

[Shahir @TslShahir]

I am talking about my long term obsession - ternary models. Expecting a banger discussion. Less than 6 hours to go

[Kautuk | Conscious Engines @Kautukkundan]

Paper Club day today! at @spc_india See you at 6!

Our SLM obsession curated this week's itinerary:

- 1 Bit LLMs w/ BitNet @sabeshbharathi

- Ternary models w/ CAT-Q @TslShahir

- extreme compression w/ TurboQuant @redrodeo03

[Shahir @TslShahir]

Our tiny model(70M) is capable of calling any tool we train it on. Here it reliably calls HealthKit to first find out how much @sabeshbharathi walked that day, and then the system calculated the steps required to complete the 10K for the day. Then the model calls mapkit to find a cafe he should walk to.

[Kautuk | Conscious Engines @Kautukkundan]

Here's a quick demo of an agentic loop running offline entirely on-device on Apple Watch Series 9.
---

We have been spearheading small models and one of the biggest unlock it provides for consumer is having intelligence right at their fingertips, no cloud-round trip = faster, cheaper, private, more control.
This also unlocks a whole suite of new use-cases with product specific models - or models built specifically for and shipped alongside an app.

We now have full expertise on the entire on-device Apple Intelligence stack. We have pioneered training and running models with Apple native APIs on iPhones, Macs and Watches.

Already working with Apple DR team and some of the biggest wearable brands exploring running AI on different form factors.

If you're a team exploring this domain, we are here to help. Reach out -> follow @cengines_studio for consumer AI

[Shahir @TslShahir]

@TaylorLineman @tryminna https://t.co/Cw8JT3UzpG

[Shahir @TslShahir]

Here is the implementation.

https://t.co/HcirRzqrL2

[Shahir @TslShahir]

Anytime a new model releases, my question is usually, can it run on the Apple Neural Engine and how fast. Happy to report that I was surprised beyond my imagination. This one has flash level velocity on the ANE. Check this out.

[Mary Newhauser @m_newhaus]

Today we’re releasing GLiNER2.5.

It’s the most significant update to the GLiNER architecture ever!

GLiNER2.5 is built on a new boundary prediction architecture that changes how the model reads text: instead of checking every possible span, it predicts where entities begin and end, which removes the limit on entity length and lets inference scale to full documents.

These changes bring five new capabilities to GLiNER users:

-Long-context extraction and classification
-Unlimited span lengths
-Joint information extraction
-Constrained classification
-Span attributes

And to explain each of these features, we’re making this week ✨GLiNER2.5 Week✨

I'll be dropping a demo video each day this week (starting tomorrow) to show you how this works on real data.

For more on the model (inc. architecture, benchmarks, use cases) see our launch blog. Big congrats to @urchadeDS and the rest of the @fastinoAI team on a terrific model!

Blog: https://t.co/0S0fSSe2CC

[Shahir @TslShahir]

Here is the implementation.

https://t.co/HcirRzqrL2

[Shahir @TslShahir]

Anytime a new model releases, my question is usually, can it run on the Apple Neural Engine and how fast. Happy to report that I was surprised beyond my imagination. This one has flash level velocity on the ANE. Check this out.

[Shahir @TslShahir]

@TaylorLineman @tryminna Directly through CoreML. @tryminna looks interesting. Will check it out

[Conscious Engines • Studio @cengines_studio]

Running custom SLMs alongside Apple Foundation models on iOS 27

Uncle Rudy is a to-do app with opinions. Models that you never see do most of the work. The app does not use one large assistant model for all tasks. It uses a set of product specific models. Each model does one job only. We selected or trained each model for that job. A fine-tuned Qwen3-0.6B model changes your speech into structured to-dos. Apple's Foundation Model reads photos of checklists. It also writes the insults from Rudy. A sentence embedding model matches each task to a sticker. All models operate on the device. No data goes out of the phone. This post shows each model, the job it does, and why a small specific model was better than a large general one.

1. Yap Model - Qwen3-0.6B (fine-tuned)

This model changes one spoken sentence into to-do actions. It is a small Qwen3-0.6B model, tuned with LoRA. It operates on the Neural Engine through Apple's CoreAI framework. The app does not bundle the model and a Background Assets extension downloads the correct model for the chip in the device.
The app gives the model the current date and time, and the five most recent to-dos. The model finds each intent in the sentence. It makes one tool call for each intent. These tool calls create, update, or delete a to-do. The sentence "buy milk also call mom" makes two tasks in one pass. If the sentence is not a to-do, the model does nothing. It does not make an incorrect task. Dates are in the ISO-8601 format. The model finds the date from the given current time. It does not do calendar calculations.
We are internally testing an even smaller ~200M parameter model to replace this.

2. Snap Model - Apple Foundation Model (vision)

This model changes a photo of a checklist, a whiteboard, or a note into to-dos. First, a Vision OCR pass reads the full-resolution photo. Small handwriting is not clear in a smaller image. Then the app sends the image and the OCR text to the on-device Foundation Model in one pass. The text gives the correct words for each task. The image shows the layout and the items that have a check mark. Guided generation makes sure that the output is a typed list. The app shows each item when the model completes it. The model session starts before the user takes the photo. This decreases the wait time.

3. Apple Foundation Model (language)

This model writes the voice of Rudy. It writes the nagging notifications, the refusal messages, and the unwilling reactions when the user completes a task. The device makes each line locally. Each line refers to the real task title. The personality setting (touch love, sarcastic, or full rudy) controls the tone. If the model is not available on a device, the app uses prepared lines. Rudy always has a voice.

4. Dynamic Profiles and NLP

The app applies the personality of Rudy through dynamic profiles in the Foundation Models framework. A profile builder selects the instructions and the sampling temperature at run time. The selection agrees with the active personality setting - could be Tough Love, Sarcastic or Full Rudy. We also use Swift's NLP based framework NLEmbedding to pick relevant stickers based on a todo!

Conclusion

All of these models show the same pattern. A 0.6B model, fine-tuned for one narrow task, is better at that task than a prompted general model. It is also faster and smaller. Apple foundation models serve as a good substrate to build off of. They read images, and they write short text in a set voice. Sentence embeddings are an old technique. But they win when the job is only similarity. Product specific models give each feature the correct intelligence for it. The full app can also operate in airplane mode. Your to-dos stay in your pocket.

[Shahir @TslShahir]

Bonsai 1.7b 1 bit is one of our favorites for on-device deployment. It is essentially magic that a ~200mb model can pack so much intelligence. This was just a short run, and we didnt push it hard enough. I am sure, we can make it even faster.

[sabesh 📟 @sabeshbharathi]

We were able to speed-up Bonsai by >2.5x!

@TslShahir trained a DSpark block drafter for @PrismML 's Bonsai-1.7B - the ternary 1-bit model, target via its bf-16 unpacked weights

> 350M params, 5 layers
> the target's residual stream at 5 tap layers gets projected and injected as K/V
> no existing recipe covered a ternary target
> rebuilt from the DFlash paper (arXiv 2602.06036)
> speedup -> 2.65x on code, 2.29x on JSON, 1.00x on prose

[Shahir @TslShahir]

Follow-up to our DSpark port: it turns out draft models take LoRAs beautifully.

We froze LiquidAI's LFM2.5-1.2B drafter and trained a ~1M-parameter LoRA on it — 4.4K JSON-task examples, regenerated by the target model, one epoch on a $1.29/hr A10.

Accepted drafts on that task went 2.37 → 2.96 per round: +17.5% end-to-end generation speed wherever speculation runs. Off-task accepts dipped slightly (2.14 → 1.89), which routing makes irrelevant — pick the adapter per request, like LoRAs on the main model, one level down the stack.

Launch to teardown: 39 minutes. Total cost: $0.85.

One base drafter, a drawer of purpose adapters, routed per task. That's the shape we're building toward.

Our ANE drafter port: https://t.co/wLn4D2JXde
LiquidAI's DSpark release: https://t.co/oqSX2fWyb6

[Liquid AI @liquidai]

The speedup varies by architecture. LFM2.5-8B-A1B is the strongest case on the H100 and the weakest on device, at 1.18x mean on the M4 Max (90 → 106 tok/s). That traces to how mixture-of-experts models currently run on Metal in llama.cpp, and to the memory the target model needs while verifying draft tokens. We publish the numbers as measured, and it is the subject of subsequent work.

The DSpark draft model checkpoints are available now on Hugging Face.

Read more: https://t.co/HHVuu0fSju
> LiquidAI/LFM2.5-1.2B-Instruct-DSpark: https://t.co/2W7lAvffxu
> LiquidAI/LFM2.5-2.6B-DSpark: https://t.co/z8BBGupluL
> LiquidAI/LFM2.5-8B-A1B-DSpark: https://t.co/usgnONMXRa
> SGLang: https://t.co/jpnDIBZVnw
> llama.cpp: https://t.co/HxEiT6DK3S

4/4

[Shahir @TslShahir]

We took LiquidAI's new 1.2B DSpark drafter and ran it on the Apple Neural Engine, while Metal keeps verification. The drafter is the rare decode component shaped like prefill: fixed 9-token block and no KV state. Drafts 2× faster, output byte-identical, +39% on a loaded Mac.

Write-up: https://t.co/ArpkgisCcr

@liquidai @c_engines

[Shahir @TslShahir]

https://t.co/jiJkv79jz3

[Shahir @TslShahir]

https://t.co/jiJkv79jz3

[Shahir @TslShahir]

@cengines_studio Love the Snap feature. Brilliant piece of design and engineering.

[Conscious Engines • Studio @cengines_studio]

If you're on iOS 27 Beta, this one's for you.

This year's WWDC announcement was the biggest one yet for Apple Intelligence, and we got obsessed with iOS 27 right after the live stream ended.

We had just one question: how completely can you adopt iOS 27? Not "does the feature run" - but how much of the platform can one real product actually use, properly, in daily life?

---

So we set out to build the most complete iOS 27 tech demo to showcase our obsession. Uncle Rudy.

→ Core AI + MLX + Neural Engine: On-device intelligence (product-specific models)
→ Foundation Vision Model: Visual intelligence to
→ Siri AI + App Intents: perform app actions from wherever you are
→ Screen Awareness: Siri acts on what’s already in front of you
→ Dynamic Profiles: three personalities of the app, one standard
→ Spotlight Semantic Index: on-device memory and app actions surfaced everywhere
→ Extra-Large Widgets: *wink wink*
→ Live Activities, Dynamic Island, Control Center, Action Button: It’s always one action away

And underneath all of it: a product-specific model. ~200M params, trained with MLX, shipped through Core AI, running on the Neural Engine. That's what we're building at the studio

Small models trained for specific jobs, owned, on your device. Uncle Rudy. Coming soon.

Drop a comment if you're already on iOS 27 Beta 6

[Retr0 @Retr06421]

@Kautukkundan @TslShahir @c_engines @sabeshbharathi Inspired by this, got a local SLM running 100% offline on a Galaxy Watch4 Classic (2021)

Running Liquid AI LFM-2.5 (350M Q4_K_M) . Taking ~340 MB active RAM and hitting peaks of ~2 t/s.

Crazy to see older wearable hardware pulling off local SLM inference.

[Shahir @TslShahir]

@Kautukkundan @clearlysid @c_engines @sabeshbharathi Yes. We can run a battery of tests tomorrow. Plus, we should do a full suite of battery measurements across all kinds of ANE on different devices. Watch, iPhone across models and Macs

[Shahir @TslShahir]

70 M ternary model running on the Neural Engine of an Apple Watch series 9. Had a lot of fun training, retraining and tuning the model. Tech report soon @c_engines

[Kautuk | Conscious Engines @Kautukkundan]

We ran an SLM fully on an Apple Watch!
(the first watch app ever to ship intelligence built into it)

My team is obsessed with the question "how small can models go and still be useful?" So @TslShahir trained a custom model which takes advantage of Apple's Neural Engine hidden inside every Apple Watch.

Full autoregressive decoding, close to 30 TPS, ON YOUR WRIST (Series 9)!! 75MB package = 70M param model.

Use-Cases?
purposefully fine tuning this model to get personal context, health data, run a tiny agent agent for some automation etc. This is exactly what will we be doing next! "Watch out" :P

(Also @sabeshbharathi exploring alt career options as a hand model)