Ivan Chan

CTO/Cofounder at @RunLocalAI (YC S24). Helping engineering teams ship better on-device AI faster and without the hassle. 🇭🇰 / 🇬🇧

[Ismail Salim @IssySalim]

🔥 VLMs on mobile devices with world-facing cameras key for proactive, intelligent computing. Local/on-device inference key for real-time, private experiences.

Great to see an emphasis on smaller VLMs. Excited to see where @huggingface, @moondreamai, etc. take things 🚀

[Ismail Salim @IssySalim]

Snap's announcement about their on-device text-to-image model seems to have slipped under the radar…

Apparently, it generates 1024x1024 images with quality that's comparable to cloud-oriented models like Stable Diffusion XL.

But it can do that locally on an iPhone 16 Pro Max in <1.5 seconds! 🤯

Snap are planning to ship it soon to their ~450m daily active users, and I wouldn't be surprised if it's free.

I wonder how all these subscription-driven, cloud-based image generation apps will respond…

Announcement: https://t.co/peEaehyqBO

Paper: https://t.co/sSD3xfuWBV

[Ismail Salim @IssySalim]

Short but sweet talk about the WebNN API: https://t.co/8EOUVOs2vm

Def worth checking out the YouTube playlist from @jason_mayes WebAI Summit last year. It's packed with great talks!

Looking forward to the next summit!

[Ismail Salim @IssySalim]

Awesome work from @soldni and team at @allen_ai!

If you're interested in shipping on-device AI language features in your app, I highly recommend checking out their demo app to get a sense of what's possible these days on an iPhone: https://t.co/cS7Yl1ZkBJ

[Ivan Chan @ivanchanavinah]

@flat https://t.co/BwY0JLGjzV

[Ivan Chan @ivanchanavinah]

@flat fantastic!

[Ivan Chan @ivanchanavinah]

will it ever be feasible to run a 10M+ token prompt using a local LLM even the LLM supports it?

[Ivan Chan @ivanchanavinah]

@awilkinson NotebookLM

[Ivan Chan @ivanchanavinah]

@Andreadful_ @ivanfioravanti @cognitivecompai This is great! Finally some detailed evaluations

[Ivan Chan @ivanchanavinah]

@omarsar0 I think NotebookLM does CAG, and it’s far better than any RAG solution I’ve ever used

[Ivan Chan @ivanchanavinah]

@ivanfioravanti @cognitivecompai https://t.co/PP85LDAg9m

this might be a relevant paper!

[Ivan Chan @ivanchanavinah]

@flat Sorry I meant *best* inference time (not highest)

[Ivan Chan @ivanchanavinah]

Let me get back to you once I’m with my computer :) but I think you’re right, because we do compression “sweeps” all the time. And usually, the pruning configs with the lowest accuracy have the highest inference time. So yes, it sounds like higher ratios lead to faster inference times

[Sam @SamuelCrombie]

Hello, world!

[David Lieb @dflieb]

Everything I wish I knew about retention before building Google Photos and Bump.

[Ivan Chan @ivanchanavinah]

@juliarturc this is amazing!

[Julia Turc @juliarturc]

There’s so much OSS out there, and I’m like a kid in the candy store. I want to try it all, but there are only so many hours in the day. Sometimes I just want to ask some high-level questions before deciding whether to invest in a repo.

We made Code Sage to scratch that itch: fresh knowledge about OSS repos, with Perplexity-like references. DM me if you want a particular repo supported!

[David Lieb @dflieb]

I already take @Waymo for granted, but I just found this photo from 2012 of the first time I saw one one the road in Mountain View.

12 years of persistence and hard work.