ilmarihei

Playing with computers.

[ilmarihei @ilmarihei]

LLM inference feels like a Chinese-style industry. Undifferentiated white-label product that requires a heavy infra buildout, lots of power, and runs on minimal margins. Probably why the labs are pivoting towards the fashion business model - differentiation through design & brand

[ilmarihei @ilmarihei]

@Paul_Kinlan Could put the oats in as well

[ilmarihei @ilmarihei]

3D printed some silver.

[ilmarihei @ilmarihei]

HW prices are going to the moon. Why? Because your salary is not going down. Every optimization adds virtual workers to the HW, each is paid human-equiv salary. Either your salary goes down or price of HW goes up. What's your hot take?

[ilmarihei @ilmarihei]

@julianharris Session switching, they've got good big KV cache systems. Maintaining a cache hierarchy HBM-RAM-Flash with optimized page-in & page-out. I guess this kind of systems eng wasn't very interesting for local inference folks earlier & needs HW to test. https://t.co/hmXtedYbRQ

[ilmarihei @ilmarihei]

Ooh, having all the text rendered is super cool. Makes me want to resurrect my ancient tabletree io. It can render the file tree but the text comes on-demand.

[Rik Arends @rikarends]

51 million lines of code. Chromium codebase. Here you go. 120hz, 3gb ram use.

[ilmarihei @ilmarihei]

@webprofusion I'm reckoning 1kW avg load here in the Tropic of Cancer, with 8000-9000 CNY for 15 kWh LiFePO4 unit. Yeah, you'd need more panels.

[ilmarihei @ilmarihei]

A 800W solar panel costs about $100. If I buy 4, and a 15kWh battery for $1600 total, that's electricity paid for the next 20 years? The current bill is $100/mth, so it would pay itself off in a year and a half.

[ilmarihei @ilmarihei]

I don't really understand business, or jobs, or investments, or money. Gonna be so cooked

[ilmarihei @ilmarihei]

Turn your MP3 into a QR radio.

[ilmarihei @ilmarihei]

Added some features to qr-send dot com to live stream video and audio over QR codes.

[ilmarihei @ilmarihei]

.. started thinking about how computers were built to run code with 16kB active datasets, but now the programs have 16GB active datasets so everything is slow. An agent will figure out how to get SOTA results out of the cache pyramid

[ilmarihei @ilmarihei]

@rikarends Uh... Why is the memory use for a 200MB dataset on a 33MB display surface 128 GB?

[ilmarihei @ilmarihei]

Instead of buying hardware, have you tried asking AI to think thoughts that make your hardware vibrate new compute units into your existing hardware?

[ilmarihei @ilmarihei]

Let's build for the near future where codegen goes at 100 MB/s, dropping a fresh 1 MB application every frame. What's needed? Verification, sandboxing, fast utils, machine code gen, ephemeral deployment, codegen based on sensor data rather than prompts. Anything else?

[ilmarihei @ilmarihei]

Generate @twominutepapers (well, 20minutepapers) out of arXiv papers with AI https://t.co/iQQKjtoiBN

[ilmarihei @ilmarihei]

Chill

[ilmarihei @ilmarihei]

LLMs, the thing limiting the model's usefulness is the human in the loop. If it takes you an hour to iterate a single prompt, you can only do 8 tasks per day.

[Moodspire @HelloMoodspire]

In the mood for chocolate? https://t.co/SwA1tjwpqS
Get your sweet slideshow fix and feel good.

#chocolate #mood #feelgood