Elizabeth Warren says her wealth tax will "rebuild America's middle class." Amusingly enough, that's true! The reason the middle class has been shrinking is not because more people are becoming poor, but because more are becoming rich. Her tax will reverse that trend.
I'm looking forward to giving an overview of Azure Quantum Elements at 3:15p PT on Mon Aug 14 in the Expo Theatre at the #ACSFall2023 meeting. Learn more about Elements here: https://t.co/UXs09N1byo
They know that the best way to solve a problem is to avoid it altogether. They are like water, which flows around obstacles and finds the path of least resistance. They are the masters of their craft, yet they remain humble and open to learning.
"The best programmers are not always the most productive. They are the ones who are able to let go of their ego and embrace the Tao. They write code that is simple and elegant, yet powerful and efficient. They understand that the best code is the code that is not written.
GPUs are built with more memory bandwidth, but higher latency and lower capacity than CPUs. AI accelerators could usefully make a different memory trade — the bandwidth of a GPU (or more), but the capacity of a CPU, in exchange for even higher latency.
For inference, all the weights, potentially a couple TB, can be accessed in a completely linear manner. It could even be a single transaction, streaming at a constant speed into an L2 ring buffer for a GPU core to chase calculations in, akin to racing the beam on old CRT game architectures. You could build a memory system out of masses of dirt cheap RAM, fully in parallel.
Even for training, the memory access patterns can be just a forward read of the weights, a reverse read , and a staggered reverse write of gradients and weights. You could have minimum transaction sizes in the megabytes, and first byte latencies in the many microseconds.
Observation suggests that people are switching to using ChatGPT to write things for them with almost indecent haste. Most people hate to write as much as they hate math. Way more than admit it. Within a year the median piece of writing could be by AI.
One of the things AI coding assistants are showing us is how much duplication there is in the code that's out there. Most people just write the same stuff over and over.