Dias Jakupov

Building secure, private, AI-native journaling app

[Dias Jakupov @PlimmApp]

Plimm is a journal that talks back. 0 of your entries leave the phone.

on-device Qwen via llama.cpp. same model on android and ios. no account, no streaks.

airplane mode it. you can still write, still talk, still get insights.

https://t.co/JYL4IJnCGp

[Dias Jakupov @PlimmApp]

an AI that reads your diary and ships it to a server is not a journaling app, its a data pipeline with a nice font. i spent three months making mine run on the phone instead, and it cost me metal, core ml and a 437mb download. go ask yours where the model runs.

[Dias Jakupov @PlimmApp]

@rivuchakraborty worth splitting the two before rewriting the ui - startup is class loading plus jit warmup, steady-state is the jit itself. native image fixes the first and often makes the second worse without pgo. if startup is the real complaint, appcds is a far cheaper experiment first.

[Dias Jakupov @PlimmApp]

@anemll what keeps me off coreml in a shipping app isnt numerics, its deployment - a second artifact per model plus the first-run ane compile stall on the users device. does the wide accumulator path change that compile step at all, or is it the same mil pipeline underneath?

[Dias Jakupov @PlimmApp]

@Tech2Wild the number that decides single-spark fit isnt the weight size though - nvfp4 shrinks weights, kv stays whatever dtype you set, and at long context thats the half that grows. worth putting weights + kv at your real context length in the recipe, not just the download size.

[Dias Jakupov @PlimmApp]

@Youssofal_ the phone end of that curve moves differently - tok/s stopped being my bottleneck a while ago, residency took over. 458mb rss with whisper loaded, 672mb with llama, never both. mtp is the thing i cant have down here, no room to hold a draft head next to the weights.

[Dias Jakupov @PlimmApp]

@techNmak the detail that matters is per-request routing with no shared cache - a multi-turn agent that lands on a different node next turn re-prefills its whole prefix. fine for fan-out jobs, rough for long chats. sticky-by-conversation would beat the load signal here.

[Dias Jakupov @PlimmApp]

@onthexitter69 @JoshuaSWarren 0.2ms is nothing for my shape and fatal for yours. whisper dispatches once per take so i never feel it. an instruction accelerator pays it every op, and the launch becomes the work. what stopped me was the other cost - first-run compile on device. did black magic reach that?

[Dias Jakupov @PlimmApp]

@dankuntz true until someone turns on voiceover. the accessibility tree, dynamic type and reduce motion all come free from the view layer and none of them exist at the calayer level - you end up hand-rolling all three. fine for the fun stuff, expensive for a whole app.

[Dias Jakupov @PlimmApp]

@WescheNex1q the 3-bit win tracks with what i see far lower down the stack - once the quant is per-tensor dynamic, average bpw stops predicting quality. i ship qwen2.5-0.5b q4_k_m on phones and the k-quant mix beats uniform 4-bit at the same size. is the 2-spark gap interconnect or exl3?

[Dias Jakupov @PlimmApp]

i shipped a journaling app with no streak counter, no reminders, no account. every journaling app i quit, i quit on a streak - day 34, miss one, and it tells you youve broken something. turns out that is the whole retention model and i didnt want to build it.

[Dias Jakupov @PlimmApp]

@NilCoalescing the accessibility edge here is worth a line - matchedGeometryEffect still animates under Reduce Motion unless you read the environment and swap to a snap. a highlight sliding across a segmented control is exactly the motion that setting exists for.

[Dias Jakupov @PlimmApp]

@iamlukethedev press-and-hold is the easy half. the one that bit me was interruptions - a call or the app backgrounding mid-take silently truncates it unless you rotate the buffer on every interruption, not just on a timer. worth testing before someone loses a long one.

[Dias Jakupov @PlimmApp]

@android_poet the bit id want to see is how the widget extension side is wired - ActivityKit lives in a separate target from the app, so it cant link the shared framework the way the app does. do consumers hand-write the attributes struct in swift and you just drive updates from commonMain?

[Dias Jakupov @PlimmApp]

@TonyZ278264 @Prince_Canuma cant answer for experts first-hand - dense 0.5b here, no MoE. mechanism should hold: mmap'd pages only hit VmRSS once touched, so it tracks what you read not what you mapped. watch VmSwap alongside - compressed pages drop out of RSS, so flat can mean reclaimed not cold.

[Dias Jakupov @PlimmApp]

@kyfriedfella worth timing the second load of the same model, not just the first - with 256gb the pages stay in cache so the reload should be nearly free, and if it isnt the cost is in the loader not the memory. thats the number we can never get, ours gets evicted before we can ask twice.

[Dias Jakupov @PlimmApp]

@Bandrew @trymirai the number id want next is acceptance rate - spec decode gives you the headline on predictable text and collapses back toward the target model on high-entropy output. and the draft model costs resident memory, which is free on an M5 Max and the entire budget on a phone.

[Dias Jakupov @PlimmApp]

@SpencerGBull @intel @OmarchyLinux fair - i argued throughput and power is the better axis. ours is bounded though: transcription runs once after the take, not continuously, so the cpu is busy just over a quarter of the audio length then idles. always-listening is where offloading stops being optional.

[Dias Jakupov @PlimmApp]

@ddalcu @pidotdev @Alibaba_Qwen @bijanbowen @Beamsters1 the iOS ceiling is the bit id ask about - jetsam kills a foreground app well below installed RAM, so shippable model size is bounded by that, not the spec sheet. i see ~672mb peak rss with the llm resident and still gate at 3.5gb. what are you hitting on MLX Chat?

[Dias Jakupov @PlimmApp]

@Hikari_07_jp the catch is that most local setups still pull weights from a hub on first run, so they break in exactly the outage theyre meant to survive. i pin sha256 and ship the download as a first-launch step for that reason - after that its genuinely offline.