Tarek Allam Jr, PhD

⚙️Lead Research Engineer @turinginst & Research Engineering Fellow @SoftwareSaved 💻 #compression , #edgeAI & #tinyML aficionado 🗜

[Anirudha Majumdar @Majumdar_Ani]

What an amazing resource for anyone interested in robotics!

For years, I've been looking for a textbook that covers the fundamentals of robot autonomy. For my robotics course at @Princeton, I used a mishmash of different textbooks for motion planning, feedback control, SLAM, and robot learning.

The new book Principles of Robot Autonomy from @drmapavone, @DanieleGammelli, Joseph Lorenzetti,
@katielulula, and @GioeleZardini is incredible! It goes all the way from the foundations of planning, control, and SLAM, to modern robot learning (imitation + RL).

I am using this as the primary reference textbook for my Intro to Robotics course at Princeton.

Book: https://t.co/hKyz7d2IeN
Lectures: https://t.co/ZpCVHrHPmP

[Tan Minh Nguyen @TanNguyen689]

In our recent paper accepted at NeurIPS 2024, my student, Rachel Si Yan Teo (@rachtsy_), uncovered a link between self-attention in transformers and kernel PCA.

Specifically, Rachel demonstrates that self-attention projects its query vectors onto the principal component axes of the key matrix in feature space. As a result, she finds that the value matrix V in self-attention captures the eigenvectors of the Gram matrix of the key vectors. She then incorporates the robust PCA algorithm into self-attention to improve the model's robustness against data contamination.

I personally find this new perspective on attention fascinating, as it opens up opportunities to explore and develop transformer models through PCA and other clustering or dimensionality reduction techniques.

https://t.co/10EQxUR5Td

[Vaibhav (VB) Srivastav @reach_vb]

On-device AI framework ecosystem is blooming these days:

1. llama.cpp - All things Whisper, LLMs & VLMs - run across Metal, CUDA and other backends (AMD/ NPU etc)
https://t.co/4hJeFOtx7c

2. MLC - Deploy LLMs across platforms especially WebGPU (fastest WebGPU LLM implementation out there)
https://t.co/ITkUU46CtK

3. MLX - Arguably the fastest general purpose framework (Mac only) - Supports all major Image Generation (Flux, SDXL, etc), Transcription (Whisper), LLMs
https://t.co/5qn4gEWpoB

4. Candle - Cross-platform general purpose framework written in Rust - wide coverage across model categories
https://t.co/SJUlZ5UDnj

Honorable mentions:

1. Transformers.js - Javascript (WebGPU) implementation built on top of ONNXruntimeweb
https://t.co/IE0EqoEbGX

2. Mistral rs - Rust implementation for LLMs & VLMs, built on top of Candle
https://t.co/4QSivT0dxm

3. Ratchet - Cross platform, rust based WebGPU framework built for battle-tested deployments
https://t.co/LXhWSJv3SP

4. Zml - Cross platform, Zig based ML framework
https://t.co/mysIAnVgli

Looking forward to how the ecosystem would look 1 year from now - Quite bullish on the top 4 atm - but open source ecosystem changes quite a bit! 🤗

Also, which frameworks did I miss?

[Charlie Marsh @charliermarsh]

Today, we're shipping a series of features that move uv beyond a pip alternative, and into an end-to-end solution for managing Python projects, command-line tools, single-file scripts, and even Python itself.

A single, unified tool. Like Cargo, for Python.

It's very fast.

[Polars @DataPolars]

🤗 x Polars!

Polars now supports native reading from @huggingface datasets. Check out our latest blog to learn more about it: https://t.co/o1oZNLBXnY

[nic lane @niclane]

TinyTrain allows *learning* not just inference to be performed on MCUs in a compute-, memory- and data-efficient manner! Young and other co-authors are at #ICML2024, reach out to chat.

TinyTrain work by @YoungDKwon1, @ruiruiliii, @SIVenieris, @jag_jagmohan, @niclane7 and @cecim

[Ritchie Vink @RitchieVink]

The first beta pre-release of Polars 1.0 is released!

Install it with `pip install -U --pre polars ` and give it a spin. If you find any troubles in the coming week, we can fix them for 1.0.

Release notes: https://t.co/90REwXu00h

The upgrade guide will follow later next week.

[Polars @DataPolars]

Hugging face just merged Polars integration 🐻‍❄️

[Lars Bergstrom @larsberg_]

Check out our new article about Google's growth in Rust usage during 2022! We debunk some common myths about Rust and share our experiences. Spoiler alert: it's faster to learn than you think, and the productivity and code quality are amazing! #rustlang https://t.co/D0e1AYBPXO

[Gautam Kamath @thegautamkamath]

Proceedings PDFs for @icmlconf accepted papers are now available on OpenReview: https://t.co/vGlDgJhfLY #ICML2023

[Vaibhav (VB) Srivastav @reach_vb]

Want to train your own Bark/MusicGen-like TTS/TTA models? 👀

The SoTA Encodec model by @MetaAI has now landed in 🤗Transformers!

It supports compression up to 1.5KHz and produces discrete audio representations. ⚡️

Model: https://t.co/Hq8rDHBfjw
Colab: https://t.co/MaWVEAMCXs

[Sanyam Bhutani @bhutanisanyam1]

Run 13B model on an iPhone! 🤯

Just finished reading @Tim_Dettmers’ amazing work on SpQR.

SpQR unlocks 3.35 bit quantisation which lets us run 33B models on 3090s and 13B models on an iPhone.

Here are my notes from the paper:

- Quantisation is basically like compressing the LLM weights

- These are normally trained at 16 or 8 bits and show performance loss at 4 bits

- An interesting insight is LLM parameters are not equally sensitive to quantisation

- Only 1% of the weights contribute to majority of the loss

- So if we can store these in 16-bits and compress everything else. We have lossless compression

- Following the above we can achieve lossless 4.75 bit and performant 3.35 bit performance.

This further gives a 4x size reduction of models and 20% speedup in inference. The best part-this happens post training.

Tim has shared both incredible work for during and post training quantisation. I’ll link his discussion in 🧵

Here’s the paper which goes into a detailed discussion of the algorithm and approach:

https://t.co/faQUEZKIPx

[Neural Compression Workshop @ICML23 @neural_compress]

The 2nd iteration of the "Neural Compression: From Information Theory to Applications" workshop will take place @icmlconf in Hawaii this year!

Submissions due May 27th. For more details:https://t.co/w5OI0jkoXl

@BerivanISIK @YiboYang @_dsevero @karen_ullrich @robamler @s_mandt

[Liam Brannigan @braaannigan]

New blog post on @DataPolars: the output from .unique is no longer set by the order of the input. Don't let it take you by surprise!

In this post I explore what happens, what's changed and why things work the way they do: https://t.co/9uWLX1h0ox

[Fink broker @FinkBroker]

Deadline for abstracts is today! Come share your work and perspectives for transient and variable astronomy in the era of Rubin!

[Polars @DataPolars]

#Python #polars 0.14.27 is released. https://t.co/5s2HHlcGk7

This release launches the alpha version of our streaming engine.

Need to process 250GB on you laptop? Try:

my_query().collect(allow_streaming=True)

Next we release we will be able to | the result to different files!

[James Baker Battles the Pink SPARQL and QS Robots @j_w_baker]

Applying to become a @SoftwareSaved Fellow was one of the best decisions of my career: it is such an amazing community that cares deeply about open research practices, software in and as research, and rewarding software/data aligned careers. If you like all that, do take a look.

[Richard McElreath 🐈‍⬛ @rlmcelreath]

Statistical Rethinking 2022 Complete Course is complete. 20 lectures from foundations of causal & Bayesian inference through DAGS, multilevel models & poststratified causal effects to Gaussian processes, Bayesian imputation & ODEs. Theatrical trailer: https://t.co/zadmoRUNmf

[Kevin Patrick Murphy @sirbayes]

Interested in converting the theory of "Probabilistic Machine Learning: Advanced Topics" into beautiful #JAX code? Available to work this summer? Consider applying to #GSoC. Details at
https://t.co/9CYF35sxa8

[Google DeepMind @GoogleDeepMind]

Today in @nature, with @EPFL, the first deep reinforcement learning system that can keep nuclear fusion plasma stable inside its tokamaks, opening new avenues to advance nuclear fusion research.

Paper: https://t.co/UdIIfNgaap