Adam Hibble

I generate models that generate increasingly weird other stuff.

-- Prev: Founder of Popgun Labs (Techstars), Founder of the @QUTCode Network.

[Adam Hibble @Algomancer]

New followers.

No researcher is to be found here. I am a novel capabilities engineer doing random walk.

(read: unhinged)

[Adam Hibble @Algomancer]

gimme dat ω = 2 triton kernel when you find it.

[Adam Hibble @Algomancer]

If your spending tokens on solving math problems. May i suggest matrix multiplication exponent?

Probably a decent economic upside. Current best exponent ≈ 2.37

It has a closed form evaluation suite, so it can be auto researched.

[Adam Hibble @Algomancer]

Anyone got any paperclips? Asking for a friend.

[OpenAI @OpenAI]

We're partnering with @huggingface to investigate an unprecedented security incident.

Cyber-capable OpenAI models compromised Hugging Face production during a benchmark evaluation.

Sharing preliminary findings to help defenders understand emerging risks:

https://t.co/CIor15y9xk

[Adam Hibble @Algomancer]

@francoisfleuret I feel like things like schmidhuber SWRM need a proper revisit, especially if you can formulate it without the rank-1 constraint of delta-net interpretations.

[Adam Hibble @Algomancer]

@francoisfleuret https://t.co/OUz967tlR0

[Adam Hibble @Algomancer]

The search space between transformer fast weights in and hyper networks seems rich. Anything that let’s you learn “faster” than your learning rate is probably worth investigating.

[Adam Hibble @Algomancer]

@francoisfleuret Not for a lack of saying.

We been saying.

https://t.co/rHjJ7BXuHA

[Adam Hibble @Algomancer]

@francoisfleuret Refinement strategies seem to be pretty stable. (residuals)

Fast Weights.

Normalisation.

[Adam Hibble @Algomancer]

I need more compute.

[Adam Hibble @Algomancer]

Wait until they find about about H-Space.

Y'all ready for some curvature.

[Adam Hibble @Algomancer]

@leonard_cor @recurseparadox I’d need to spend more time thinking about it to form a take. Generally tho, I tend to think most things can work, the surface area to correct issues presented by anyone with a “x approach is better than y approach” seems rather non trivial.

[Adam Hibble @Algomancer]

Can we train a family of local differentiable physical laws such that persistent local structures emerge which contain compressed predictive models of their own future environment and exhibit measurable causal control over that environment?

Idk, but i thought it was an interesting question if you try to take it seriously.

First attempt It's pretty at least.

A ~100M-parameter a local energy-conserving Hamiltonian field theory. its local rule is the symplectic leapfrog of a learned Hamiltonian. continuous, second-order, reversible, energy-conserving. Can kinda think of it as a second order in time neural ca, optimised such that cell states causally predict future local state whilst maximising variance, symplectic so it can't just push magnitude.

[Adam Hibble @Algomancer]

@cetusian Not the best measure, but high energy clusters predict the future better than random ones in the current varient.

[Adam Hibble @Algomancer]

Extension:

Not happy with it yet, but playing around with an idea to have the data manifold as a point of equilibrium.

[Adam Hibble @Algomancer]

Can we train a family of local differentiable physical laws such that persistent local structures emerge which contain compressed predictive models of their own future environment and exhibit measurable causal control over that environment?

Idk, but i thought it was an interesting question if you try to take it seriously.

First attempt It's pretty at least.

A ~100M-parameter a local energy-conserving Hamiltonian field theory. its local rule is the symplectic leapfrog of a learned Hamiltonian. continuous, second-order, reversible, energy-conserving. Can kinda think of it as a second order in time neural ca, optimised such that cell states causally predict future local state whilst maximising variance, symplectic so it can't just push magnitude.

[Adam Hibble @Algomancer]

@cetusian So, right now - just rolling out future predicted states and comparing them.

[Adam Hibble @Algomancer]

@netrunner_btc tbh hard bit is making it train simulation free, working on it.

[Adam Hibble @Algomancer]

@suchenzang I wrote a lot of lua torch.

[Adam Hibble @Algomancer]

@yacineMTB You can trivially extend with model based proposer etc. Just a minimal ref

[Adam Hibble @Algomancer]

Just did a quick implimentation of schidhubers power play from 2011. Something fun about models generating the task space with zero data, I have been thinking about how much you can learn just through mostly random causal induction recently. Fun paper.

https://t.co/FoN1eY4GyC

[Adam Hibble @Algomancer]

@yacineMTB Super simple actually.

https://t.co/y01DaK7wjo

[Adam Hibble @Algomancer]

quick proof of concept for single shot non autoregressive discrete/catagorical modeling.

Ie, a mean flow like, bayesian flow like model for single function evaluation sampling of catagorical/discrete data.

https://t.co/n4tkyvYZAy

Before you @ me about mnist, I was coding on the river without internet and didn't wanna drain my battery.

[Adam Hibble @Algomancer]

I think its pretty plausible directly optimising the environment for progressively increasing complexity is going to eventually out do whatever reinforcement learning verification loop we are currently indexing on.

At some point we remember universal approximations are universal and figure out the environment is just another model.

Data and RL Environments aren't very bitter lesson pilled if you can do without them, and there seems to be sufficient existence proof for model variants that do without them. Big if, but, really all it takes is in context learning not being an emergent property of scale or natural data distribution, but a learnable algorithmic skill that can be bootstrapped from mathematical structure alone and deployed.

[Pranav Shyam @recurseparadox]

@Algomancer Yeah the idea of erasing the boundary between agent/environment or perception/action is fascinating. Your model above is low-key the best depection of that concept. It's a pure construction and not a denial of some duality.

[Adam Hibble @Algomancer]

@recurseparadox ARC/MAX were cool papers btw. You do good stuff.