James Melvin

AI, Entrepreneur

[James Melvin @jamesmel24]

@neurosp1ke All the best 🚀

[James Melvin @jamesmel24]

@neurosp1ke Is this on unity ?

[James Melvin @jamesmel24]

@neurosp1ke Very cool 😎 🚀

[Nando de Freitas @NandoDF]

A model trained for next-token prediction is forced to build compressed representations of latent structure in text. Ilya Sutskever correctly refers to this phenomenon as understanding. Here, a model trained for next-step sensor prediction, with a robot that has proprioception and touch sensors but no vision, is forced to build compressed representations of latent structure in the physical world. The robot becomes aware of the shape of external objects. That is, it understands the physical properties of the external world that enable it to make better next-step sensor predictions.

This research was previously done by a diverse team of expert engineers at DeepMind over a month - including stars like @notmisha and @yuvaltassa. Remarkably, this reproduction with a completely different robot took only a few hours to implement using Codex.

The automatic creation of physical environments by AI will likely lead to huge advances in areas of science and engineering that use physical simulators or twin models.

The paper and notebook are available at ❤️∀
https://t.co/z2N29JF1FY

[James Melvin @jamesmel24]

@neurosp1ke Very cool 😎

[James Melvin @jamesmel24]

@neurosp1ke There should be something like Bluetooth SIG or 3GPP where the protocols are standardised

[James Melvin @jamesmel24]

@douwekiela @FelixHill84 RIP

[Andrej Karpathy @karpathy]

I gave a talk at GPU MODE workshop last week on llm.c

- the origin story of llm.c
- being naked in the world without PyTorch and having to re-invent Array, Autograd, Device, Dtype, Compile, Distributed
- how to port a PyTorch layer to 1) explicit PyTorch
- and then to 2) write the backward pass
- 3) port forward & backward pass to C
- 4) string all the layers together
- achieving one file of C with no dependencies that compiles and runs ~instantly, where all memory is pre-planned and allocated a single time, fully deterministic, portable code that can run on a potato or a von Neumann probe
- how most of llm.c was built at 1am-7am in a water villa porch in Maldives and why this is the recommended way to develop software
- convert all of it to run in CUDA on GPU in fp32
- port matmul to cuBLAS
- port attention to cuDNN flash-attention
- introduce bfloat16 mixed precision
- introduce many more optimizations and features like kernel fusions, Packed128, stochastic rounding, full determinism
- add multi-GPU training, NCCL, sharded optimizer
- add multi-node with MPI or file system or socket
- reproduce GPT-2 (1.6B) on one 8XH100 node in 24 hours for $672 in llm.c, achieving (at the time) 29% less memory, 19% faster training that PyTorch nightly, and much faster compile & run
- how open source development attracts Avengers from the internet
- port to training Llama 3 imminent (branch exists)
- many other notable forks
- last thought: how software abstractions like Python/PyTorch and everything else really exist only because humans are finite in knowledge, IQ and attention, and how with increasing AI capability LLMs may export custom binaries like llm.c for any application directly, tearing apart and refactoring all abstractions as needed.
<|endoftext|>

More links in reply

[Junda Chen @Junda_Chen_]

That’s us! It was so much fun and adrenaline in the hackathon! Thanks @Accel and @marksaroufim @neurosp1ke @msharmavikram for organizing this unforgettable event. Especially thanks to @__tensorcore__ for all the technical deep dives. Stay tuned for our work on FlexAttention/FA3

[PyTorch @PyTorch]

We’re happy to officially launch torchao, a PyTorch native library that makes models faster and smaller by leveraging low bit dtypes, quantization &amp; sparsity. Our techniques are written in easy-to-read PyTorch code spanning both inference &amp; training: https://t.co/jzSL2RrH2C

[Andreas Köpf @neurosp1ke]

It was amazing to meet so many incredibly talented CUDA hackers in real life. My brain still needs to recalibrate to what is possible here in the Bay Area 🇺🇸💙.

[Andreas Köpf @neurosp1ke]

We will talk about @_chris_lu_ &amp; colleagues' paper:

`The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery`

Sat, Aug 17 @ 6 PM UTC

Join in on @ykilcher's discord: https://t.co/BEGebaqHBe

[Andreas Köpf @neurosp1ke]

OpenThought is a new initiative for cognitive architectures (agents), system-2 reasoning, self-improvement and general problem solving.

Let's together compile the best strong AI material list:
https://t.co/C7UT47ndEw

Chat: "#open-thought" on Yannic Kilcher's discord

[Andreas Köpf @neurosp1ke]

Currently planning H2/2024 for CUDA-MODE.

We start with a look at accelerators of NVIDIA competitors &amp; WebGPU:

Jul 20 (Today): AMD - Speaking Composable Kernel (Haocong Wang)
Aug 17: Intel - SYCL-MODE (Patric Zaho)
Aug 24: WebGPU gpu.cpp (Austin Huang)

[Andreas Köpf @neurosp1ke]

Recording: https://t.co/HW7XajpVyb
Slides: https://t.co/N54Zpg3eVJ

[Andreas Köpf @neurosp1ke]

CUDA-MODE today:
Speculative Decoding

Tokens go brrr via drafting and verification...
Cade Daniel is big time vLLM contributor and the original author of vLLM's speculative decoding impl.

7 pm UTC
June 1, 2024

👉Session details: https://t.co/1Cz9lwwi3g

[Jeremy Howard @jeremyphoward]

There's a new bill, SB-1047 "Safe and Secure Innovation for Frontier Artificial Intelligence Models Act".

I think it could do a great deal of harm to startups, American innovation, open source, and safety. So I've written a response to the authors: 🧵
https://t.co/1aUrHnAjrF

[Andreas Köpf @neurosp1ke]

Lecture 16: On Hands Profiling
Video: https://t.co/EzKCzD5005
Look over the shoulders of CUDA profiling guru Taylor Robie as he analyzes PyTorch code using various profilers (compute & memory). 📊🚀

[James Melvin @jamesmel24]

@Apple adds more LLM tech and its open-source 😎

https://t.co/OiW1HTjEhk

[Jeremy Howard @jeremyphoward]

Today at @answerdotai we've got something new for you: FSDP/QDoRA. We've tested it with @AIatMeta Llama3 and the results blow away anything we've seen before.

I believe that this combination is likely to create better task-specific models than anything else at any cost. 🧵