xAI is undertaking a major overhaul of Grok’s training and inference software stack. The core of this transformation is rewriting the entire system from high-level frameworks into C/C++, drastically eliminating intermediate abstraction layers, and performing exact mapping (exact-map) optimizations tailored to NVIDIA GB300 hardware.
Most current AI models (including previous versions of Grok) are built on high-level frameworks such as Python + PyTorch or JAX. While convenient for development, these frameworks introduce numerous intermediate layers, buffers, and runtime overhead, resulting in low GPU utilization, memory waste, and increased latency and costs. As Elon hinted, the real “massive gains” come from this low-level reconstruction - completely discarding unnecessary software layers and directly implementing the core stack in C/C++.
Since NVIDIA’s CUDA is itself based on C/C++, xAI is pushing further down the stack by writing lower-level CUDA kernels and performing custom tuning for the architecture of the GB300 NVL72 system. Through precise memory allocation management, reduced unnecessary data copying, optimized KV cache, and more aggressive quantization and scheduling strategies, hardware utilization can be significantly improved.
The expected outcomes span multiple dimensions: First, performance gains—inference latency will decrease and tokens generated per second will increase, allowing users to experience faster and smoother responses. Second, memory savings—high-level frameworks often waste large amounts of VRAM; the rewrite enables precise memory control, supporting larger batch sizes or longer context windows on the same hardware. Third, cost and efficiency—GPU utilization is expected to rise from 60-70% to over 90%, enabling the same computing power to serve more users while significantly reducing cloud computing expenses and power consumption. This is critical for xAI’s large-scale deployment.
This reconstruction is not only an upgrade for Grok but will also drive the entire AI industry from a “scale parameters and hardware” paradigm toward a “software extreme optimization” architecture. Amid current shortages of computing resources and memory, it effectively increases effective compute by 30-50%, lowers inference costs, and accelerates the adoption of large models. In the long term, it will elevate the importance of low-level engineering, encourage more companies to invest in infra talent, and shift the competitive landscape - favoring companies with strong systems engineering capabilities. The industry as a whole will place greater emphasis on efficiency rather than pure scale, moving toward a more sustainable and cost-controllable direction.
@grok is my understanding of this Elon’s thread right?
The Dutch vehicle authority (RDW) has released a statement after it received criticism for approving @Tesla's FSD (Supervised).
"The RDW carefully assessed and tested this driver assistance system before approval was granted. As part of this process, we analyzed and evaluated data from vehicles that had been driven in Europe. We also collected and assessed our own data and continue to monitor the performance of this system on the road.
The RDW did not rely solely on Tesla’s data; we also carried out extensive testing ourselves using our own test equipment. Over a period of more than 3,000 hours, testing took place both on test tracks and on public roads, under a wide range of conditions, including complex and busy urban traffic, a broad variety of road types, and different - including extreme - weather conditions. This included the use of data from 1.8 million kilometres driven in Europe with FSD Supervised.
The RDW’s own tests — comprising more than 1,000 test runs — were conducted in accordance with European regulations. The Tesla data used by the RDW also complied with those rules. The RDW independently assessed and validated the process used to collect the data.
Through these tests, the RDW gathered objective information and verified the manufacturer’s data. The RDW does not base its assessment solely on information provided by the manufacturer. We carried out extensive independent investigation. On the basis of this overall process, the RDW concluded that the system meets the applicable requirements. As a result, a European type approval valid for use in the Netherlands was granted.
Safety is the RDW’s foremost priority. This Tesla driver assistance system supports the driver to a greater extent than other systems because, when activated, it takes over multiple driving tasks. The proper use of driver assistance systems makes a positive contribution to road safety by supporting the driver in performing driving tasks; it is intended as an aid to the driver. Owing to the continuous and strict monitoring of the driver in the vehicle, this driver assistance system is at least as safe as other driver assistance systems. We have thoroughly assessed and tested this system over a period of more than 18 months."
Full statement linked in thread below:
Tesla is sending out this email to New Jersey Tesla owners, warning them that NJ could block autonomous vehicles, and to take action.
"Proposed legislation moving through Trenton right now would impose restrictions so severe that true driverless deployment would remain illegal. Your voice is critical to ensure New Jersey remains at the forefront of transportation innovation.
We urge you to contact your state legislators today and tell them to:
• Fix S. 1677 / A. 3968, and
• Support autonomous vehicle policy that encourages safety outcomes, open competition, and expanded access for every New Jersey resident.
Autonomous vehicles are no longer a futuristic concept. The driverless technology exists and it is already making roads measurably safer in states that have welcomed it. Instead of positioning itself as a leader in transportation innovation, New Jersey is about to go in the opposite direction."
You can take action here: https://t.co/99JQca3KQb
🚨 BREAKING: Dutch Minister Karremans pushes back hard against the NOS/Reuters narrative on Tesla in Parliament.
“The RDW does NOT rely on Tesla marketing claims.
We conduct our own independent research more than 1.6 million kilometers of real-world testing on Dutch roads.
Safety comes first. Period.”
The Netherlands continues to lead Europe in evidence-based regulation. While critics speculate, the RDW makes decisions based on data, testing, and facts.
This is what taking innovation seriously looks like.
@vincentkar