The 2nd SAIR Competition is now open: the Modular Arithmetic Challenge.
Can a neural network learn to compute (a × b) mod p for large numbers?
Easy for standard algorithms, but a hard test of whether neural networks can generalize the computation.
Join the SAIR competition:
https://t.co/SL2uhfVuJD
@reidhoffman.— LinkedIn co-founder, PayPal Mafia, OpenAI board, MSFT board, Greylock partner.
One of the most influential operators in Silicon Valley sits down with SAIR co-founder Chuck Ng. Science x AI Summit, May 12.
Register: https://t.co/k9V9qPiH41
Math is everywhere but it's invisible. Happy π Day!
SAIR co-founder Terence Tao reminds us that the everyday tech we rely on is powered by hidden math. Take a moment today to notice the math around you!
#PiDay#TerenceTao#Mathematics
People often ask me why we think Databricks can succeed in new areas we expand to. This is the recipe why: we build stellar teams in areas where we think we can greatly improve on the status quo. Lakehouse was one, Lakebase is next, but there's more coming, especially in AI.
We disclosed today as part of our Series L that our 4-yr old data warehousing business is now >$1B revenue run rate. This is to the best of my knowledge the fastest to $1B DW product in the industry. How did we do it, and what’s next?
The conventional wisdom is that it would take 5+ years to build a new database (just to release one).
Four years ago, the linked blog announced that Databricks had won the official TPC-DS 100TB benchmark with DBSQL, which was in preview back then. It had the best perf and the best price/perf, and notably beating Snowflake by 12x in price/perf in that benchmark. (Note: we are still the top place on the official TPC-DS benchmark today.)
That blog post launched a contentious "benchmarking war" with a lot of back and forth between vendors, but more importantly it marked the very beginning of our data warehousing business.
To build this business, we assembled the best engineering team and established a new infrastructure product category called Lakehouse that inherits the flexibility and openness of data lakes and performance of data warehouses. Lakehouse is now the standard for data infrastructure, and organizations are migrating from legacy data warehouses to the Lakehouse.
The result so far is a testament to the team and their execution. We have a lot of ideas on how to take performance and usability to the next level, and the team is working hard to make that happen. Expect some big announcements next year. We want to lay the foundation for growing the data warehousing product to a $10B business.
Databricks had operated largely in the “analytics” side of data in the past, and we believe the “operational” side of data (aka “OLTP”) is also ready for a “Lakehouse” style disruption. A huge chunk of the founding team’s time is now focusing on “Lakebase”, a new category of OLTP databases that separates storage (in the lake) from compute. That architecture enables features that have been virtually impossible for databases in the past: instant provisioning, elastic scaling (down to zero), branching, high throughput scan directly from Spark, …
I won’t go into too much detail about Lakebase here, but we expect a similar trend to happen in the next few years: Lakebase will transform the industry and other OLTP systems will re-architect or position towards it.
The best data warehouse is a lakehouse, and the best database is a lakebase!
https://t.co/BHrOE18Gtt
We did a very careful study of 10 optimizers with no horse in the race. Despite all the excitement about Muon, Mars, Kron, Soap, etc., at the end of the day, if you tune the hyperparameters rigorously and scale up, the speedup over AdamW diminishes to only 10% :-( Experiments are made possible by Marin (https://t.co/UgEjGM0HPY); anyone developing new optimizers: please come try your method on this benchmark!
I have decided to teach a new course in this Fall at Hong Kong University: Principles of Deep Representation Learning based on a new manuscript that my colleagues and students are preparing on Learning Deep Representations of Data Distributions. This would be a first course that attempts to study deep learning, and a significant part of intelligence, from the first principles.
The National Academy of Engineering is excited to announce the appointment of Tsu-Jae King Liu as its next president! A trailblazing researcher, innovator, educator, and academic leader, Liu will begin her six-year term on July 1, 2025. Read more: https://t.co/Z1nJoNolr6
🚀 Really excited to launch #AgentX competition hosted by @BerkeleyRDI@UCBerkeley alongside our LLM Agents MOOC series (a global community of 22k+ learners & growing fast). Whether you're building the next disruptive AI startup or pushing the research frontier, AgentX is your launchpad. Two tracks:
- Entrepreneurship: Build agent-powered products & startups
- Research: Explore the frontiers of LLM Agents technology
📅 Registration opens TODAY! Submissions due end of May
🏆 Winners showcase at our Agents Summit to industry leaders and VCs in August @UCBerkeley! 🌟
🙏 Tremendous thanks to our incredible sponsors @Amazon@huggingface@LambdaAPI@MistralAI@Google@GroqInc@schmidtsciences; proud to partner w. leading VCs in the space @Accel@BainCapVC@BessemerVP@lightspeedvp@MayfieldFund@NEA! Stay tuned—more sponsors/partners AND exciting prizes/credits/resources info will be announced soon! 🚀
⏰ Register now at https://t.co/1tXZOB2BVL and join us in shaping the future of AI! #AgentX #AI
Biggest Ever UC Berkeley Funding Round: Databricks, founded @UCBerkeley, raised a landmark $10B funding round at a $62B valuation. https://t.co/ndkITJ7HDh
Trump says the CHIPS Act and Science Act are "so bad" and should be replaced by tariffs.
Wondering whether @elonmusk and all thoe Silicon Valley MAGA supporters will appreciate tariffs on Taiwanese GPUs 😅
.@UCBerkeley recently hosted the nation’s first-ever Asian American, Native Hawaiian and Pacific Islander Higher Education Leadership Development Summit, sponsored by the White House #AAPIHeritageMonth https://t.co/Rkc0MLfpER
.@UCBerkeley recently hosted the nation’s first-ever Asian American, Native Hawaiian and Pacific Islander Higher Education Leadership Development Summit, sponsored by the White House #AAPIHeritageMonth https://t.co/Rkc0MLfpER