nicknegulescu

Digital Editor of Online Encyclopedia: The Cascade District.
sea gulls, sirens and jet engines. artisanal tweets.

[International Space Station @Space_Station]

The International Space Station orbited over Los Angeles on July 4th as America marked 250 years of independence with a burst of fireworks lighting up the city below — a celebration so bright it reached all the way to space!

[Shawn Farash @Shawn_Farash]

This is Socialism.

This is what Maduro did to his people in 2019.

He had opposition protesters RUN OVER by armored military vehicles.

This is what the Democrat Party is defending.

This is what they want to do to you.

Do not let them win.

[Jeremy @Jeremybtc]

The scammers are evolving 💀

[nicknegulescu @nicknegulescu]

@seattledot what happened to express lanes this afternoon? Someone fall asleep? Directions of the express lane are closed

[Tanishq Mathew Abraham, Ph.D. @iScienceLuvr]

DeepSeek released an OCR model today.

Their motivation is really interesting: they want to use visual modality as an efficient compression medium for textual information, and use this to solve long-context challenges in LLMs.

Of course, they are using it to get more training data for their models as well.

"DeepSeek-OCR can generate training data for LLMs/VLMs at a scale of 200k+ pages per day (a single A100-40G)."

[Andrej Karpathy @karpathy]

I quite like the new DeepSeek-OCR paper. It's a good OCR model (maybe a bit worse than dots), and yes data collection etc., but anyway it doesn't matter.

The more interesting part for me (esp as a computer vision at heart who is temporarily masquerading as a natural language person) is whether pixels are better inputs to LLMs than text. Whether text tokens are wasteful and just terrible, at the input.

Maybe it makes more sense that all inputs to LLMs should only ever be images. Even if you happen to have pure text input, maybe you'd prefer to render it and then feed that in:
- more information compression (see paper) => shorter context windows, more efficiency
- significantly more general information stream => not just text, but e.g. bold text, colored text, arbitrary images.
- input can now be processed with bidirectional attention easily and as default, not autoregressive attention - a lot more powerful.
- delete the tokenizer (at the input)!! I already ranted about how much I dislike the tokenizer. Tokenizers are ugly, separate, not end-to-end stage. It "imports" all the ugliness of Unicode, byte encodings, it inherits a lot of historical baggage, security/jailbreak risk (e.g. continuation bytes). It makes two characters that look identical to the eye look as two completely different tokens internally in the network. A smiling emoji looks like a weird token, not an... actual smiling face, pixels and all, and all the transfer learning that brings along. The tokenizer must go.

OCR is just one of many useful vision -> text tasks. And text -> text tasks can be made to be vision ->text tasks. Not vice versa.

So many the User message is images, but the decoder (the Assistant response) remains text. It's a lot less obvious how to output pixels realistically... or if you'd want to.

Now I have to also fight the urge to side quest an image-input-only version of nanochat...

[Tim Durkan Photography @timdurkan]

A fun little electrical storm rolled through Seattle tonight… you might say it was an electrifying experience ⚡️

[nicknegulescu @nicknegulescu]

Seems legit

[nicknegulescu @nicknegulescu]

Driving around Seattle, the smoothest driving surface road in the city is over the Ballard Bridge, including the span. @dongho_chang 😉

[nicknegulescu @nicknegulescu]

@TiffanyMSmiley @PBS @NewsHour Just watched it! Superb job! Thank you!

[Future 42 @future42org]

The Ferry System is still broken, but now WSF is staring down a potential 19% budget cut due to WA State's refusal to follow Federal Law.

@deniseonKOMO at KOMO

[nicknegulescu @nicknegulescu]

At Yalta, U.S. President Franklin D. Roosevelt, British Prime Minister Winston Churchill, and Soviet Premier Joseph Stalin made important decisions regarding the future progress of the war and the postwar world.

https://t.co/8uP3Bf7GYx

[nicknegulescu @nicknegulescu]

"The motivator is largely the same: AI systems, and the crawlers that power them, may undercut the economic interests of anyone who publishes content to the web—by using the websites’ own data."

https://t.co/GAmd5dWieL

[KOMO News @komonews]

The U.S. Geological Survey is closely monitoring Mount Adams after six earthquakes were recorded in September, the highest amount since record-keeping began in 1982. https://t.co/og0KdomAzd

[Mindful Maven @mindfulmaven_]

MIT University just released free online courses.

No payment is required.

Here are 7 courses you don't want to miss in 2024:

[nicknegulescu @nicknegulescu]

Developers, Developers, Developers

[jess @_jessicasachs]

dev rel circa September 2000

there's an entire generation of devs who have never seen Steve Ballmer sweating through his dress shirt during a Microsoft keynote screaming "developers".

let's fix that. also if you've seen it here's the full video with context.

[David Fowler @davidfowl]

I *love* switch expressions #dotnet #csharp

[NWS Seattle @NWSSeattle]

Sequim is being Sequim today. #wawx

[City of Abbotsford @City_Abbotsford]

!!! EMERGENCY UPDATE: The Cit The City of Abbotsford is issuing an immediate & urgent notice to all residents that did not yet evacuate Sumas Prairie to evacuate immediately. Conditions within tSumas Prairie within the last hour have escalated & pose a significant risk to life.

[Wheels (Sean) @wheeler244]

First Snow! ❄️ Washington Pass Liberty Bell #wawx and a Rainbow 🌈 . A mix at the pass with 37° @RandySmall @ShannonODKOMO @ScottSKOMO @ErinMayovsky @WaWxChasers @NWSSeattle 🥶