LiveFromVR

Making VR Games and videos of stuff I find in Virtual Reality. AI Enthusiast

[LiveFromVR @LiveFromVR]

@RonenV but generally speaking yes, agree

[LiveFromVR @LiveFromVR]

@RonenV It actually wasn't made with AI

[LiveFromVR @LiveFromVR]

@gdechichi Relax, relax! Only until it gets decompiled and there is a fully featured PC version.

[LiveFromVR @LiveFromVR]

@the_aiju Second probably only to...San Diego, which I've never been to but I've heard is wonderful

[LiveFromVR @LiveFromVR]

@the_aiju It has some of the best weather of any city in America year round. Truly a charmed city.

[Taelin @VictorTaelin]

Since our pitch deck was uninformative, let me try to explain our whole thesis. Sorry if any part of it sounds obvious.

I argue that any reasonable definition of AGI must imply a system capable of learning as efficiently as a human. LLMs obviously don't pass this bar. When you talk to them, they don't learn anything, since the weights don't change. So, unless true infinite context lands, or GPUs become 10000x cheaper, the current methods won't scale to AGI, at least in that sense.

(Note that this has nothing to do with raw capabilities. Perhaps Fable 10 will be building Dyson spheres, yet still be unable to internalize new knowledge as efficiently as humans do. So, one might ask if we even *need* AGI in the sense above. I think we do, if only because a drastic drop in pre-training costs would make AI more democratic, and prevent one of the worst outcomes in my mind: over-concentration of power.)

As you know, modern AI is based on neural networks. What not everyone knows is that NNs are just very, very slow programs. Nobody in their right mind would sort a list using NNs, because that'd be millions of times slower than just using C. So, why do we implement *intelligence* with NNs?

As a thought experiment, imagine if we could take Fable's weights and distill them, not into a smaller model, but into a raw C program that was observationally identical to it. Literally, a C file with astronomically complex, tangled procedures that, when compiled and executed, behaved just like Fable 5.2.

If such a thing were possible, then we could have Fable at a million tokens per second. And here's the cool part: such a thing *is* possible. At least, as far as computer science is concerned, nothing says it isn't. So, why don't we do just that?

Because we don't know how to *train* a raw program.

Neural networks are slow, but they're differentiable, so we know how to train them. Step by step, slowly, but surely. Raw programs are fast, but we have no methods capable of training them. So we don't. We just stick to NNs, and accept that it takes $50 billion to train a model. We take that as "normal". But is it?

In theory, it shouldn't be like that. As the old saying goes: first you make it work, then you make it fast. We know AI works. Now, we will make it fast. That's the premise behind Symbolic AI. HOC's first milestone is to train a model comparable to GPT-2, with no NNs. If that works, we might have the fastest inference engine in the world. Not by 2x, but by 1000x.

So, in a way, HOC is like DeepSeek or Jev, in the sense that we're all about efficiency, but in a much more extreme way. People are aiming for hardware, but we believe there are still 1000x gains to extract in the software layers, if we're willing to rethink the whole stack from literal scratch.

Many laugh and say this is impossible, but they say so with no basis. They just observed that many failed to build symbolic AI models in the past, and concluded it is impossible, or very hard. That's not a formal proof. In fact, there are many reasons people in the past could've failed, yet the actual problem may end up being easier than most anticipate. Symbolic AI flourished half a *century* ago. Back then, they didn't have:

- As much knowledge as we do

- Massive GPU clusters

- Methods to scale Symbolic AI (Bend3, HVM5!)

So, just like everything that once seemed impossible, there's one core prerequisite to making it possible: *trying*.

That's what we'll do!

I don't think I'm smarter than all the brilliant researchers who worked on this problem in the past, and failed. I just think I'm equipped with tools, compute, and knowledge that they didn't have. So, standing on the shoulders of giants, and with enough dedication, I don't think it is so unlikely we stumble upon some symbolic algorithm from which, given enough compute, unexpected capabilities emerge.

If it does work, I do think business is straightforward. At the very least, we would be able to sell incredibly high quality datasets to big labs, but, at that point, just selling inference might work as well. Business is obviously not the wall, it is all on whether such massive software-based speedups exist, and whether we're able to discover them.

[LiveFromVR @LiveFromVR]

@dccommonsense I will. That clip was so good, a surprise at every turn. What wit.

[LiveFromVR @LiveFromVR]

@luckeyfaraday It's a hard pill to swallow but it's true. The end of the world in some ways. But also the beginning of a new one.

[LiveFromVR @LiveFromVR]

Ok this tweet finally got me to go try it, wow it actually is very impressive. This is going to be very popular soon.

[Kevin Pereira @Attack]

Decided to finally poke at @spawn.

~30min later, I've applied to become a partner and have inquired re: investment opportunities.

Hashtag not-an-ad in the slightest. I'm just very, truly, incredibly, impressed.

Curious if anyone who sees this is building as well?

[LiveFromVR @LiveFromVR]

Probably my all time favorite tweet

[kache @yacineMTB]

you're trying to be 2 steps ahead of me and then you notice i'm 4 steps behind you. i wasn't going that way, bitch, keep up

[LiveFromVR @LiveFromVR]

@Liv_Boeree Not everyone! I myself am a bright eyed (perhaps naive) optimist. Even the things I see as big problems I'm pretty sure will get worked out just fine.

[LiveFromVR @LiveFromVR]

@DragonsEyeImage @davidhudnall That cool looking. It always seemed like you could do a lot with a wrap like this on a cybertruck

[LiveFromVR @LiveFromVR]

@davidhudnall lmao somebody actually did it https://t.co/DC06Ig3wHt

[LiveFromVR @LiveFromVR]

@signulll Cybertruck with some kind of novelty wrap

[LiveFromVR @LiveFromVR]

Rewriting a whole new Linux in Bend sounds really hard at first, and it will be. But way you could start is by only writing the drivers that you need. Drivers in general can probably be written more or less on the fly going forward. We can still merge them in too.

[LiveFromVR @LiveFromVR]

@rileybrown Thanks so much for sharing this. I feel like I learned a lot reading your prompts. I think I need to be way more iterative than I have been with 5.2 so far

[LiveFromVR @LiveFromVR]

@SyntheticBeef Could be that the new models get so good that a single sentence prompt produces something more impressive than Star Wars. Or no prompt.

Thats probably not months away but maybe years. I think people kind of know this even when they insist they don't. And it freaks them out.

[LiveFromVR @LiveFromVR]

@SyntheticBeef Skills definitely do something. You can tell that whatever people knew before AI is what they're really good at getting the AI to do, if they learn how to use the AI at least. But how long does that even last? I'm not quite sure to be honest. Maybe these are the last months of it