The problem with Anthropic's consciousness paper
My last post got more attention than I expected, and the question I keep getting is some version of "okay, so what is actually wrong with the paper?". Let me try to explain.
First, the core result is fine. Reading out intermediate-layer representations and asking which ones the model can actually use downstream is a real question, and people have been poking at it for years. The J-lens is a reasonable tool. If you strip the paper down to the linear algebra, it is a decent piece of interpretability work.
My problem starts one level up. This did not need to be a paper about consciousness. It did not need global workspace theory, it did not need the brain, and it did not need the word "conscious" anywhere near it. The same experiments, the same figures, the same tool, all survive perfectly well as plain interpretability. Someone chose to wrap it in neuroscience. That choice is the product, not the science.
At the end, this is the main thing. Almost everything Anthropic ships as blogs/papers/posts is PR. They build genuinely good models and they are even better at packaging them.
A publication used to carry a specific kind of weight. People spent years on something and wanted to tell the world what they found. It happened mostly in academia, with a few industrial labs as the exception, Bell Labs, IBM and Google (for a while), but the distance between the paper and the product was real. When you read a paper you could assume the authors were not trying to sell you something underneath the ideas. There were outliers, but they were the minority, and the researchers you trusted would not risk their name on a narrative.
We are not in that world anymore. Every startup now publishes blogs and papers to raise its visibility, and that is fine, that is marketing and everyone knows it. Anthropic does something more effective. They erase the line between legitimate research and PR. We get confused because the models are so good, so we assume the outputs are research. A lot of the time they are selling us something. Sometimes it is "our models are safer," sometimes it is "our models are more capable," sometimes it is positioning for regulation. The consciousness framing serves a narrative they already committed to, models that look more and more like the brain, from a lab that has publicly tied itself to AI welfare and moral patienthood. The direction of the push is not subtle.
If you want the sharper version of the technical objection (disclaimer: I'm not an expert) Global workspace theory is a theory of access, not experience. Ned Block's distinction between access consciousness and phenomenal consciousness exists precisely to block the inference this framing invites. Access tells you nothing about whether there is anything it is like to be the system. The paper is careful enough to say it demonstrates no subjective experience. But that disclaimer is not what propagates. What propagates is "consciousness" in the same sentence as "Claude," published by Anthropic, borrowing the vocabulary of neuroscience to lend biological weight to a subspace of activations. The paper keeps the rigor of Block's vocabulary and drops the rigor of his argument. Most people who see the headline will never read either one.
Of course a lab named Anthropic is going to anthropomorphize its models. But we should be able to separate a good interpretability tool from the story it is dressed in.
So that is my problem. Not the math. The narrative bolted onto the math, and our willingness to keep calling it research.
We’ve sponsored Tailwind CSS 💙 @tailwindcss.
Tailwind is foundational to our frontend, and as an open source project ourselves, this is our way of supporting the open source software we rely on.
#opensource#opensourcecommunity#developers
If you're a designer then you should be racing to get as close to the product as possible so that you can experience your design decisions as you make them and have the agency to iterate on those decisions yourself. Even the best design tools won't get you there.
🎉 Dify v1.0.0 is FINALLY HERE! 🎉
📦 https://t.co/jM7glDlNMg
We’re thrilled to announce the launch of our new Plugin system! This game-changing update decouples models and tools from the core platform, making it simpler to add features and build even more powerful AI apps.
🔑 What’s Exciting?
- Plugin System: Models and tools are now decoupled, enabling easier updates and a Marketplace with 120+ plugins, including new Agent Strategies and Extension plugins ready for you to explore!
🔗 Explore Dify Marketplace: https://t.co/ytatkKaUVR
- Smart Workflows: Meet Agent Nodes! They automate tool use and decisions in your workflows and chats.
- Open Ecosystem: Join our growing community, share resources, and easily integrate with platforms like Slack and Discord.
Dify v1.0.0 makes adding features a breeze. Developers can now effortlessly create and contribute plugins, bringing even more capabilities to AI applications.
Dive into our blog and kickstart your journey today!
https://t.co/Ko7tTFg9Qj
#AI #Dify #Plugin #OpenSource
In the final #datainthehallway episode of 2024, we sat down with Chenhe Gu (Head of DevRel at @dify_ai) to discuss #GenAI and how Dify can help #tidb community members build AI applications. You can check out the episode at https://t.co/bLImpj32uh
If you’re going to hang around rationalist spaces, you should deeply internalize what I call Hanson’s razor: never attribute to malice that which can be adequately explained by autism.
🎉 We’ve just hit 50k stars on GitHub!
❤️ A huge thanks to our incredible community for being part of this journey.
We’re about to unveil something big in v1.0, something we’ve been working on day and night. It’s set to change the game, making Dify more open and accessible than ever.
💻 For developers, adding new features to Dify will be more flexible—not just limited to PR submissions.
🧑💻 For AI app builders, Dify will soon allow easy connections to external apps and tools.
Stay with us—we can’t wait to show you what’s next! 🎊
We're thrilled to host the @TechCrunch Disrupt AI Agent Hack Day as an official #Disrupt2024 side event!
A day of epic hacking innovation, fun competition, and groundbreaking ideas is happening tomorrow and we’re pulling together an incredible day with innovative companies including @storiai_com@neondatabase@restackio@ToolhouseAI@dify_ai 🎉. It's at the @EdgeAGI Open Source Campus.
There's still time to register here: https://t.co/l9vbLjpJNR
Following @itsajchan for more coverage on this super fun and engaging day!
🚨 BREAKING: We've been selected for the prestigious #StartupBattlefield!
We're one of just 200 startups chosen from thousands to exhibit at #TechCrunchDisrupt2024. This is a huge milestone for us!
Join us in San Francisco from October 28-30. We can't wait to showcase our work and connect with you there!
➡️ Learn more: https://t.co/O1kRNzROhw
🌐 We’ve got some big news to share – DifySandbox goes open source.
DifySandbox is one of the backbones of agentic workflows in Dify.
It serves as the execution environment for user-written code while also blocking malicious code to ensure system security.
🎊 We’ve now made it open source to align with Dify’s goal of building an open and community-driven ecosystem. It offers several key advantages:
- Extensibility: DifySandbox integrates Python and Node.js, and it’s set up to incorporate more languages.
- High Performance: Its single-container, multi-task setup seriously speeds up tasks.
- Open Access: As an independent open-source project, DifySandbox allows for free use.
😎 We're excited to invite you to join our community and help shape its future.
💻 repo: https://t.co/MGRfBsbwPR
🗺️ contribution guide: https://t.co/2ibkRvsyOn
📓 blog for additional tech insights: https://t.co/y3bfZ4Qxpm
s/o @Yeuoly1
🚀 We're Hiring!
Dify is looking for a Technical Business Development Manager! If you're passionate about tech and business growth, apply now!
📍 Location: Suzhou OR Shanghai
📧 Apply: https://t.co/Mb97aUv0zV
The @kyutai_labs fully end-to-end audio model demo of today is a huge deal that many people missed in the room
Mostly irrelevant are the facts that:
- they come a few week after OpenAI ChatGPT-4o
- the demo was less polished than the 4o one (in terms of voice quality, voice timing…)
Relevant:
- the model training pipeline and model archi are simple and hugely scalable, with a tiny 8+ people team like Kyutai building it in 4 months. Synthetic data is a huge enabler here
- laser focus on local devices: Moshi will soon be everywhere. Frontier model builders have low incentive to let you run smaller models locally (price per token…) but non-profits like Kyutai have very different incentives. The Moshi demo is already online while the OpenAI 4o one is still in limbo.
- going under 300 ms of latency while keeping Llama 8B or above quality of answers is a key enabler in terms of interactivity, it’s game changing, This feeling when the model answer your question before you even finished asking is quite crazy or when you interrupt the model while it’s talking and it react… Predictive coding in a model, instantly updated model of what you’re about to say...
Basically they nailed the fundamentals. It’s here. This interactive voice tech will be everywhere. It will soon be an obvious commodity.
@kishiwadapeople sorry to hear this -- we found that this is an issue with windows file permissions and have made PRs to fix it. your issue should be gone now if you still like to give it a try
.@dify_ai x @langfuse integration released
I am super excited about this partnership as it is one of the cleanest langfuse integrations yet. Build and host LLM applications with @dify_ai, monitor, evaluate and debug them with @langfuse
congrats @guchenhe & team!