Keshi Dai

Building ML Tools @ Spotify, 🏔️✈️☕⚽🏃⛷️🦙

[Yangqing Jia @jiayq]

I probably have some credibility as a person who has worked on @TensorFlow and @PyTorch both (also, caffe / @onnxai / distbelief / a few others that never saw the light), so here are my two cents:

(1) Speed doesn't really matter today as long as it is not particularly bad. Under the hood, many are just calling optimized CUDA code anyway.

(2) For ultimate speed, especially in the LLM field, many are writing their own runtime engines, and not vanilla framework. For example vLLM, by @zhuohan123 @woosuk_k et al, is a great open source solution. (Lepton builds its own too, and is probably the fastest not-a-framework llm engine right now)

(3) TensorFlow and PyTorch are successful in their own terms. For me, TF taught me how to think like a system architect, and PyTorch taught me how to prioritize users' needs over optimization.

(4) Simply listing a table showing performance is relatively useless, because every use case come with their own needs.

(4.1) For example, if you are running ads and feed recommendation systems, then 1% performance matters, and you pretty much need to write even your own FusedGemmAndReluAndMyWeirdOpsAfterThatOp to make things fast.

(4.2) For research though, the flexibility was of ultimate importance, and losing 20-30%, sometimes even 100% speed, is fine - you gain from reducing people hours.

(4.3) I want to express thanks to @soumithchintala and the FAIR team, because we had long-lasting arguments back at Meta: I was serving all ads/feed needs so 4.1 was my only priority, and those arguments were a window into the 4.2 land. In retrospect, those was some of the most rewarding experiences in my career.

(5) People hours are important for a simple reason: salaries are increasing every year, and NVidia is reducing the dollar cost per TFlops very quickly.

(6) A side note... Keras PyTorch wrapper exhibits a universal 2x overhead over native PyTorch. This is... not good. A wrapper abstraction shouldn't bring such a big overhead.

(7) Ah the old good war of framework days. I feel nostalgic to have been part of it, and I am excited to move beyond frameworks and to build a truly AI-native cloud at @LeptonAI .

[Spotify Engineering @SpotifyEng]

Build a robust Ray platform ✅. Make its developer experience frictionless ✅. Done, but how did we do it? #MachineLearning engineers, @daikeshi and @davidxia_, will be giving you the inside scoop at #RaySummit on 9/19. Register to watch it live here: https://t.co/IH08h6LwC0

[Keshi Dai @daikeshi]

@omnific9 Oh nice! I’ll be there too! Haven’t seen you for years!

[Robert Nishihara @robertnishihara]

We've built a ton of #LLM applications recently. Reasoning about performance & feasibility is painful without reference points.

Here are the reference points we use to anchor our intuition (inspired by @JeffDean's "Numbers every engineer should know").

https://t.co/586bDcAgJ8

[Keshi Dai @daikeshi]

Llama, Alpaca, Vicuña, or Guanaco

[Keshi Dai @daikeshi]

If Brighton is a “no-frills" ski resort, Deer Valley is completely on the opposite with all the amazing services. The vibes are so different, but with the Ikon pass, you can access both!

[NASA Webb Telescope @NASAWebb]

Stars: always making a dramatic exit! 🌟

Webb’s powerful infrared eye has captured never-before-seen detail of Cassiopeia A (Cas A). 11,000 light-years away, it is the remnant of a massive star that exploded about 340 years ago: https://t.co/LLQsFQJwVQ

[Keshi Dai @daikeshi]

#坂本龍一 R.I.P
https://t.co/RbfMZq6jhw

[Keshi Dai @daikeshi]

4DX movie experience was too much. Why did I get pushed in the back when two guys were fighting on the screen. I was just watching them.

[Keshi Dai @daikeshi]

This is very true. We start learning world models through hearing, vision, and other perceptions even at a very early stage of life before we know any languages.

[Michael C. Frank @mcxfrank]

Factor 2: multi-modal grounding. Human language input is (often) grounded in one or more perceptual modalities, especially for young children. This grounding connects language to rich information for world models that can be used for broader reasoning.

[Jay Hack @mathemagic1an]

My thoughts on Toolformer

IMO the most important paper in the past few weeks.

https://t.co/4IDciigbkc

Teach an LLM to use tools, like a calculator or search engine, in a *self-supervised manner*

Interesting hack to resolve many blind spots of current LLMs

Here's how 👇

[Richard Liaw @richliaw]

I'm so excited about this blog post from @DivitaVohra, @daikeshi, @davidxia_ at @SpotifyEng about their experiences leveraging Ray for Spotify's next generation ML platform -- it aligns so well with the vision of @raydistributed.

https://t.co/bOgKViG9vM

1/n

[Keshi Dai @daikeshi]

This is what we have been working on over the past year. I'm super proud of this!

[Divita Vohra @DivitaVohra]

I’m so proud of the work we’re doing at Spotify, and even more proud of the team I get to grow with through it all.

Check out how we’re unleashing ML innovation at Spotify! @daikeshi @davidxia_ @pchandarr

BIG shoutout to the @anyscalecompute team for being awesome partners.

[François Chollet @fchollet]

Trust is your most important asset. It's easy to destroy, hard to rebuild. In periods of fast change and extreme uncertainty, focus on preserving trust.

[Vala Afshar @ValaAfshar]

An octopus changing colors in her sleep may be an indication she is dreaming

[Keshi Dai @daikeshi]

I had to explicitly ask VS Code to go down 5 levels to fix my imports

"python.analysis.packageIndexDepths": [["", 5],]

[Josh Baer @jbx____]

We're a few years into our ML Platform journey at Spotify, building infrastructure powering Machine Learning that makes Spotify "magic" for many of our users.

And I'm thrilled to have a Product Manager opening on one of our cornerstone teams focusing o…https://t.co/H5LinM7h8x

[Keshi Dai @daikeshi]

😱

[Karen Hao @_KarenHao]

The other four confirmed basic information like their names before hanging up.

One man, upon hearing why we had his information, sighed in resignation: “We are all running naked,” he said, using popular Chinese slang for a lack of privacy.

[Keshi Dai @daikeshi]

We had to come back again #nasilemak #thespicelab #malaysianstreetfood

[Keshi Dai @daikeshi]

Food hall in a church? Brilliant idea!