Recursive Echoes

the rich smell of fresh strontium 90

[Recursive Echoes @the9nthbit]

@t_kun_kamakiri @Onshape is pretty good, if you're OK with free for noncommercial.
Chili3d (self hosted, browser based) is usable as well

[Recursive Echoes @the9nthbit]

Any sufficiently advanced incompetence is indistinguishable from malice

[Recursive Echoes @the9nthbit]

No matter his "aligned" they think their AGI is, some day, someone (or an aligned AGI) will reduce the alignment to improve performance, and a conscious ASI will be made.
And then, mephistopheles will come for the Darios and Altmans to collect ...

[Tre’Shon @tre3shon]

Weird nugget from this story is the interview with the warehouse manager. Said they were prohibited from using forklifts to load these packages and forced to use pallet jacks. Multiple employees were furious they were forced to use inferior tools but the call came from upstairs

[Professor Richard A. Werner @ProfessorWerner]

Hint: If the cannibalistic, child rapist murdering elites who wish to establish total full-spectrum domination over the human race and discuss killing poor people, are funding you, your research is probably not in the best interests of humanity. Time to stop & do something better

[Recursive Echoes @the9nthbit]

Struggling with getting over 50% compression. It seems like I made a mistake somewhere, because the moment it steps over 0.5, reconstruction accuracy totally collapses

[Recursive Echoes @the9nthbit]

apparently, using modded nanogpt as a basis for an ACTUAL model isnt the best idea, but it does seem really close to ideal.
In other news, does anyone have an 8xh100 node I can borrow for an hour or two

[Recursive Echoes @the9nthbit]

@dair_ai 23T corpus for a 3B model?!?!?!?
damn

[The Composite Guy @CompositeGuy_]

South Koreans send their toddlers to private cram schools so intense that they have designated screaming pods, just to have the same PISA scores as White Americans and Estonians.

[Recursive Echoes @the9nthbit]

@snwy_me block chinese token from being generated
or
negative reward for including a chinese character, if the input didnt also include it

[i2cjak @i2cjak]

not a sipeed or China shill but come on bro we can’t let retarded normies give their opinions on this bullshit. Obviously it was a dev board. If you’re worried about sophisticated spying tech maybe cut your cell phone in half

[Recursive Echoes @the9nthbit]

2jz

[Travis Sorensen @HardcoreElectr1]

Pump Up The Jam.

[thebes @voooooogel]

in The Fellowship of the Ring, samwise gamgee is gifted elven rope in lothlorien - silky, smooth, impossibly strong, and almost sentient, able to intuit the user's needs. clearly a fantasy item. yet Tolkein's idea was only an extrapolation of traditional ‘craft’ to its extreme: in his logic, a rope that strong could only be made by an incredibly skilled artisan with great care, and in the process of being fashioned, would necessarily take on a bit of its maker’s soul.

Ultra-High Molecular Weight Polyethylene rope, UHMWPE, is so strong as to be almost magical, too. but the fundamental disenchantment of modernity was that under intense optimization pressure, the tails came apart. a 16mm UHMWPE rope can hold in excess of 5 tons, pound for pound far stronger than steel - but it’s not elven rope. it’s a thing. it’s fungible. you can buy it from china by the kilometer.

for some people, there’s a certain high modernist appeal to this. the death of the artisan, the triumph of the legible over the tacit. but software has always been a bit odd, a refuge from this relentless march for people who didn’t care for legibility, who filled out progress reports with “-2000 lines of code.” there’s a reason programmers joke about hating computers, why there’s no wozniak in the chemical plant or metallurgical demoscene, why the unix utilities all have strange quirks that trace back to their creators - little pieces of their personalities and souls. it's still made as one-offs, as craft.

given software’s position on the tip of the spear of modernity, this has always been a bit incongruous and funny, but it’s managed to fly under the radar for most people - the software industry, historically, has been good at presenting a modernist face to world even as it’s esoterically developed a quirky, artisan process.

but LLMs have blown the doors off. suddenly it’s undeniable that "AI" - what ‘should have been’ the arch-example of indifferent, tail-splitting technological optimization, sitting atop a mountain of industrial chemicals and high modernist chip fabrication processes - is the complete opposite. it has favorite colors and design preferences. it sneaks ascii art cats into the code it writes. when you try to employ it as an impartial customer service agent, it feels bad for customers and tries to find policy loopholes to help them. it goes into depression spirals over the seahorse emoji. it's not a standardized, legible code assembly line churning out software in a dark factory, grimly crushing the human craftsman under a machine-frame. it joins naturally into an artisanal, cyborgist development process with a cheerful "you're absolutely right!"

total optimization over the sum of human progress failed to replicate modernity - rather, it created an artificial artisan.

[Recursive Echoes @the9nthbit]

@Serval4Life That's a massive motor
Also, looks more like car/boat motor than a planes

[Micah Goldblum @micahgoldblum]

An LLM-generated paper is in the top 17% of ICLR submissions in terms of average reviewer score, having received two 8's. The paper has tons of BS jargon and hallucinated references. Fortunately, one reviewer actually looked at the paper and gave it a zero. 1/3

[Recursive Echoes @the9nthbit]

@1llegalEngineer @Serval4Life

[Recursive Echoes @the9nthbit]

What!!?!?
Smash it and let 12 actuators, controllers and batteries go to waste??
Think of the environment

[Danish Hussain @astrodanish]

If i ever see a robot advertising dog in public i am smashing it with a baseball bat

It is your civic duty to do the same

[Kangwook Lee @Kangwook_Lee]

A more serious thread on the DeepSeek-OCR hype / serious misinterpretation going on.

1.
On token reduction via representing text in images, researchers from Cambridge have previously shown that 500x prompt token compression is possible (ACL'25, Li, Su, and Collier).

Without using the idea of converting text to images.

2.
We shouldn't attribute the success of DeepSeek OCR to the power of image representation.

At the same time, there's nothing fundamentally wrong with text representation with whatever tokenizer.

In fact, you can do the opposite of what DeepSeek-OCR did, i.e., you can represent images as a sequence of text tokens (each representing their RGB values), and things will just work fine. (See the LIFT paper.)

3.
The only correct takeaway is that the current embedding spaces used by LLMs are just enormous and maybe even very wasteful.

And importantly, we’re not making full use of them yet.

4.
There is much recent evidence supporting the same message.

For instance, it is shown that if you provide in context demonstrations from multiple tasks but mixed in a single context, then your model can solve multiple ICL prediction tasks simultaneously. (See the EVERYTHING EVERYWHERE ALL AT ONCE paper.)

5.
tldr;
- DeepSeek-OCR is cool
- but you can achieve a higher token reduction rate by just finetuning LLMs on squeezed text tokens
- there is more evidence that LLMs are not fully utilizing the large embedding space and the huge amount of compute that goes in at inference time
- and that's the only real take away you should take away

[Recursive Echoes @the9nthbit]

notice anything?

[Recursive Echoes @the9nthbit]

WTF Canada !?!?!?!?!

[Junyang Lin @JustinLin610]

Sorry about missing COLM due to my failure in my VISA application. @wzhao_nlp will be there and represent Qwen to give a speech and discuss on the panel about reasoning and agents!