Announcing @LeadbayAI $4.3M to kill everything you're doing with sales.
Corgi, Deel, L'Oréal recently told us: "We found in 10 min on Leadbay what we couldn't in 6 months with Clay and a GTM agency."
Leadbay qualifies the millions of data-signal-scarce companies nobody could find before.
This is how it works:
After shutting down my YC company, I spent >6 months looking for a job and grinding leetcode alone, despite having interned at @nvidia and @amazon.
So I built an AI to do live mock coding interviews with and finally landed a great role.
Hopefully I’m the last solo leetcoder.
🚀 Introducing MoBA: Mixture of Block Attention for Long-Context LLMs
Excited to share our latest research on Mixture of Block Attention(MoBA)! This innovative approach revolutionizes long-context processing in LLMs by combining the power of Mixture of Experts (MoE) with sparse attention. MoBA achieves efficiency without sacrificing performance, making long-context tasks more scalable than ever.
🔑 Key features of MoBA:
🔹Trainable block sparse attention: Capable of continued training from any current full attention model
🔹Parameter-less gating mechanism: Seamlessly switches between full & sparse attention
🔹Production-proven quality at https://t.co/fiOj6qZoJN , 6.5x speedup at 1M input
🌟 Training and inference code available on GitHub: https://t.co/3F1j9ETF5u
Today we’re launching the Anthropic Economic Index, a new initiative aimed at understanding AI's impact on the economy over time.
The Index’s first paper analyzes millions of anonymized Claude conversations to reveal how AI is being used today in tasks across the economy.
@ed_delan@andrewchen I join the discord as a user, and I love the thing! Supposedly they are beta testing the ability for for you to interract with the ai podcast host and guest in realtime and record it
Adding intermediate reasoning tokens effectively adds layers to the computer graph, making the model deeper and allowing it to simulate deeper circuits. We might soon see LLM can potentially be Turing complete
What is the performance limit when scaling LLM inference? Sky's the limit.
We have mathematically proven that transformers can solve any problem, provided they are allowed to generate as many intermediate reasoning tokens as needed. Remarkably, constant depth is sufficient.
https://t.co/HO2seV73KT (ICLR 2024)
Can neural networks be Turing complete?
Traditionally, without CoT, transformers are limited to parallelizable tasks within the AC0 complexity class, solving problems computable by shallow Boolean circuits due to their fixed architecture and inherent parallelism.
CoT reasoning fundamentally changes this landscape by enabling transformers to handle sequential computations through intermediate reasoning tokens. Each CoT step effectively adds a layer to the computational graph, increasing computational depth and allowing the model to simulate deeper circuits beyond AC0.
This advancement brings transformers into the realm of P/poly—the class of problems solvable by polynomial-size circuits. With sufficient CoT steps, a transformer could, in theory, simulate any computation that a polynomial-size circuit can perform, narrowing the gap between transformers and Turing machines.
Practical limitations exist, such as finite context windows and computational resources. Leveraging this potential requires careful model design and optimization.
Extending this concept, we can envision neural networks functioning analogously to operating systems. Just as an OS manages resources and orchestrates complex processes, neural networks could dynamically control computational tasks, allocate attention, and sequence operations to solve intricate problems efficiently.
Integrating CoT reasoning in transformers offers a promising path toward Turing completeness. Despite practical challenges, this perspective could revolutionize artificial intelligence by transforming neural networks into general-purpose computational frameworks capable of complex reasoning and decision-making.
An excellent paper