New research: I've discovered a universal ~20D geometric structure in semantic knowledge across 19 transformer models (11M-7B params).
This reveals geometric limits of meaning and enables 178× compression - 6.2× beyond state-of-the-art.
Paper, code & data: https://t.co/G3xAFnxnU7
chains of thought for o3-mini!
(we try to organize the raw CoT to make it more readable, and optionally to translate languages, but we try to keep it quite faithful to the raw one)
great work from @mia_glaese, @joannejang, @akshaynathan_ , and their teams!
Video-LLaMA, a multi-modal framework that empowers Large Language Models (LLMs) with the capability of understanding both visual and auditory content in the video.
https://t.co/6igPgxyoqg #LLM#AI
AI's hunger for power is growing. Models like O3 might turn nuclear reactors into standard data center equipment. Is AI becoming the world's newest utility?
#ai#agi#nuclear
OpenAI's new model tried to avoid being shut down.
Safety evaluations on the model conducted by @apolloaisafety found that o1 "attempted to exfiltrate its weights" when it thought it might be shut down and replaced with a different model.
OpenAI's o1 preview shows incredible PhD-level reasoning at lightning speed. It solved complex electrodynamics problems that usually take 1.5 weeks—in just 3 minutes, it correctly solved three. This will lead to scientific breakthroughs soon, and this is only the o1 preview! #o1
Eric Schmidt says in the next year, AI models will unite three key pillars: very large context windows, agents and text-to-action, and no-one understands what the impact will be but it will involve everyone having a fleet of AI agents at their command
My current views:
1. Superintelligent AI is very risky and we should not rush into it, and we should push against people who try. No $7T server farms plz.
2. A strong ecosystem of open models running on consumer hardware are an important hedge to protect against a future where value captured by AI is hyper-concentrated and most human thought becomes read and mediated by a few central servers controlled by a few people. Such models are also much lower in terms of doom risk than both corporate megalomania and militaries.
3. Splitting into "small" and "large" categories, exempting small, and regulating "large", seems reasonable in principle. And 405B is larger than consumer hardware size (70B is not; I run it). But the worry is that many current proposals end up making everything slide into the "large" category over time.
After almost a decade, I have made the decision to leave OpenAI. The company’s trajectory has been nothing short of miraculous, and I’m confident that OpenAI will build AGI that is both safe and beneficial under the leadership of @sama, @gdb, @miramurati and now, under the excellent research leadership of @merettm. It was an honor and a privilege to have worked together, and I will miss everyone dearly. So long, and thanks for everything. I am excited for what comes next — a project that is very personally meaningful to me about which I will share details in due time.