This is interesting as a first large diffusion-based LLM.
Most of the LLMs you've been seeing are ~clones as far as the core modeling approach goes. They're all trained "autoregressively", i.e. predicting tokens from left to right. Diffusion is different - it doesn't go left to right, but all at once. You start with noise and gradually denoise into a token stream.
Most of the image / video generation AI tools actually work this way and use Diffusion, not Autoregression. It's only text (and sometimes audio!) that have resisted. So it's been a bit of a mystery to me and many others why, for some reason, text prefers Autoregression, but images/videos prefer Diffusion. This turns out to be a fairly deep rabbit hole that has to do with the distribution of information and noise and our own perception of them, in these domains. If you look close enough, a lot of interesting connections emerge between the two as well.
All that to say that this model has the potential to be different, and possibly showcase new, unique psychology, or new strengths and weaknesses. I encourage people to try it out!
@elonmusk Consider connectivity weighted votes? Might be interesting
Known people followers increase your weight (perhaps by a proportion of each followers own weight)
Known bots followers reduce your weight
if you follow a bot you have zero weight (community will reject bots over time)
SiGenex Inc one of 17 companies invited by Wyss Institute for Biologically Inspired Engineering Diagnostic Accelerator industrial participants programme
Harvard University HMX – Harvard Medical School https://t.co/fRn9GW6DKD
We at Sigenex are honored to have been invited by the Wyss Institute @ Harvard University to participate in the first Industrial Participant Program to accelerate diagnostics capabilities.
This is an important step in our journey…https://t.co/u06LqtRN5x https://t.co/gpJmhJuCyg