Andrew Dickson

[Andrew Dickson @xordrew]

@jxmnop i mean how bad is it if i just prepend wikipedia to every sample

[Andrew Dickson @xordrew]

For those in the niche of self-assembling structure, I hacked this simulator together https://t.co/ioCRgE8Qzw

It's a CTMC of transitions between quasi-static states in crystal growth, much like kTAM. And like kTAM it can build pretty much anything. E.g. ->

[Andrew Dickson @xordrew]

@maxhodak_ I suppose you could train a trial inclusion discriminator on early data, and treat your RCT as an evaluation of both discriminator and therapy jointly. Still clumsy, but maybe fits in the framework?

[Andrew Dickson @xordrew]

@apgox Tangentially, I am curious about how good a pseudo-replicator can get. Say a cluster of computers controlling replicating teleop robots. It’d be an awful lot safer

[Andrew Dickson @xordrew]

@francoisfleuret If we went ahead and spent 10% of the world GDP training a monolithic MLP on a million tokens of context, would it actually be bad? I’m not even sure.

[Andrew Dickson @xordrew]

@moultano And we train them to act human (and conscious) almost adversarially. Hard not to believe you'll trick yourself

[Andrew Dickson @xordrew]

@TaliaGraceSable If I want to be high effort about it, I'd say pick a number in the thousands, add up its digits, check if the ones digit of the result is less than 3.

[Andrew Dickson @xordrew]

@ZoldenGames If you don't mind saying, did you base your engine on any papers, projects, etc? I've been looking for a good starting point for a general purpose particle simulator.

[Andrew Dickson @xordrew]

@samswoora Closest I've seen to this is hypernetworks. Fun question though, if you somehow get this working, how do you avoid training on test?

[Andrew Dickson @xordrew]

@typedfemale primary google glass use case

[Andrew Dickson @xordrew]

@_AashishReddy I have, all the time. I've found gpt is maybe 80% reliable when I ask it for papers, but still has a tendency to make up titles. This is for proteomics / microbiome topics though, ymmv.

[Andrew Dickson @xordrew]

@eryney_ok Chatterjee w/ the masked diff. models

[Andrew Dickson @xordrew]

@francoisfleuret If only the reparameterization trick worked for chain-of-thought tokens.

[Andrew Dickson @xordrew]

@vikhyatk Which float type? The default eps in AdamW can cause nans for bf16 iirc.

[Andrew Dickson @xordrew]

@DdelAlamo There's a preview available through github copilot, that's probably a lot of the usage

[Andrew Dickson @xordrew]

@ducx_du Do you have proposals you like for autoregressive models in latent space? I'd love to see anything like that, but defining the log-likelihood objective (without some horrifying VAE style latents) is intimidating.

[Andrew Dickson @xordrew]

@slimer48484 @argyros_selini It's a reasonably well known problem in RL fields, e.g. https://t.co/y0Q9UqIYEF. Technically it only applies to models that purely go through pre-training.

[Andrew Dickson @xordrew]

@DamonLisch @AdrianoAguzzi @yungkingmito And it always uses the same phrases! "-the real killshot" makes my skin crawl at this point.

[Andrew Dickson @xordrew]

@ZoldenGames Whoa, I think I remember seeing this on reddit when you first developed it?