@Alexintosh@anemll Hi , just FYI I have forked your repo https://t.co/TClRPPjZed and modified repacking script to include vision tensors and modified IOS app with most of the configurable parameters in UI. This is my repo - https://t.co/eXZlDvDSpe . Again, you are doing amazing stuff, thanks for sharing your good work.
1:21 PM
@googlegemma please make iOS gpu dynamic libraries available for external parties to enable litert-lm with Gemma-4 .Currently, it seems to be working only for CPU backend when built from source to use in our iOS app
Looking for GPU accelerator on iOS for Litert-lm to build iOS app with Gemma-4. Seems there is none yet for external parties. The google Edge Gallery does run on GPU so there is one but internal to google. @googledevs@GoogleDeepMind please make it available
@osanseviero Just wondering my app in Apple Store does similar thing concept wise check this out https://t.co/dG0oVmjEUg is it co-incidence ,was working on new release and saw this Vow !!! @googledevs@googleDeepMind@sundarpichai
Just wondering if google new agents skill edge gallery app shows the same concept similar to my app on Apple Store https://t.co/HSvKRePpvQ launched one month back on using agents with tools using local model and skills. @danveloper@Prince_Canuma looks like we are on right track π haha ..
@Prince_Canuma You are correct on context length Turbo Quant , I could see benefits in longer context length but that would then hit resource constraint if running on limited resource devices
@Alexintosh@anemll@danveloper@danpacary Great work @Alexintosh and @anemll , I could run 35b a3b q4 model at 4.2 Tok/sec with k=4 and 4 chunks on my iPad Air m2 with your repo code. I see you managed to get 13 Tok/sec , are this updated in repo ? Branch ?
Another angle to discussion.I feel that can the interaction be harvested using key words that typically captures the brain or idea of individual so his/her intelligence becomes personal and used for training and this local model is used by individual.This can be attempt to digitise your brain and influence AI personal to you.
I believe that people have started believing what AI says is always correct. Need to educate our people and too much dependency that is created on AI.After seeing google ads and ChatGPT ads in India I am worried.@narendramodi@Google@OpenAI
@karpathy Vow did you use and vibe coding for any part of the code or vibe coding gave up but your brain succeeded. Yours surely show power of human brain.