AI leadership currently focussed on GenAI for customer support. @uber @aminohealth @jawbone @qualcomm.🤖Robotics PhD USC, ⚙️BTech IIT Bombay. she/her/mom.
This is a great read and commentary. For me, the last few lines really resonated: "this tumultuous weekend showed just how few people have a say in the progression of what might be the most consequential technology of our age."
The amazing @_KarenHao and @cwarzel document the goings on inside OpenAI leading to the Friday event. Sharing some hightlights with commentary below:
https://t.co/x1D5UX0S5o
@savvyRL I missed the poll, but I agree strongly. I think it’s more important now than ever to make development of AI tech and applications accessible to everyone.
Last CALL!
.@shubhi_says is hosting a Women in AI mixer on 27th May in Koramangala!
🥁🥁🥁
Please share this with your friends' WhatsApp group, post on internal Slack and Teams!
And help other folks find this — RT for Karma please?
https://t.co/dk7Bl0V1zX
@NirantK@prateeks@shubhi_says@lavanyats@zainabbawa Oh I would love to join, thanks for asking me. I can’t be in Bangalore on the 27th, so will miss this event, but would love to connect with the AI group in Bangalore. I will be there June 4th to 15th.
Public livestream event on May 19 with several leading genAI researchers.
"In an era in which convincing images, audio, and text can be generated with ease on a massive scale, how can we ensure reliable access to verifiable, trustworthy information?"
https://t.co/df1xPvWoTi
Another great resource to understand recent developments in open source LLMs.
Ahead of AI #8: The Latest Open Source LLMs and Datasets, by @rasbt https://t.co/UFrqwPcW4P
Oops haven't tweeted too much recently; I'm mostly watching with interest the open source LLM ecosystem experiencing early signs of a cambrian explosion. Roughly speaking the story as of now:
1. Pretraining LLM base models remains very expensive. Think: supercomputer + months.
2. But finetuning LLMs is turning out to be very cheap and effective due to recent PEFT (parameter efficient training) techniques that work surprisingly well, e.g. LoRA / LLaMA-Adapter, and other awesome work, e.g. low precision as in bitsandbytes library. Think: few GPUs + day, even for very large models.
3. Therefore, the cambrian explosion, which requires wide reach and a lot of experimentation, is quite tractable due to (2), but only conditioned on (1).
4. The de facto OG release of (1) was Facebook's sorry Meta's LLaMA release - a very well executed high quality series of models from 7B all the way to 65B, trained nice and long, correctly ignoring the "Chinchilla trap". But LLaMA weights are research-only, been locked down behind forms, but have also awkwardly leaked all over the place... it's a bit messy.
5. In absence of an available and permissive (1), (2) cannot fully proceed. So there are a number of efforts on (1), under the banner "LLaMA but actually open", with e.g. current models from @togethercompute, @MosaicML ~matching the performance of the smallest (7B) LLaMA model, and @AiEleuther , @StabilityAI nearby.
For now, things are moving along (e.g. see the 10 chat finetuned models released last ~week, and projects like llama.cpp and friends) but a bit awkwardly due to LLaMA weights being open but not really but still. And most interestingly, a lot of questions of intuition remain to be resolved, e.g. especially around how well finetuned model work in practice, even at smaller scales.
Leaked Google document: “We Have No Moat, And Neither Does OpenAI”
The most interesting thing I've read recently about LLMs - a purportedly leaked document from a researcher at Google talking about the huge strategic impact open source models are having
https://t.co/q2lsjTHKGS
Top ML Papers of the Week (April 24 - 30):
- AudioGPT
- Track Anything
- Agents Learn Soccer Skills
- Harnessing the Power of LLMs
- Scaling Transformer to 1M tokens
- A Cookbook of Self-Supervised Learning
...
"patience, attention to detail, and thinking deeply about small things".
Unless you are building throwaway demos, these three things make the execution of non-trivial ML projects appear as if there is no progress when "progress" involves doing those three things.
My team at Uber is hiring ML engineers and researchers in SF/Bangalore/Hyderabad. We work on automating and improving customer support using Conversational AI (chatbots, speechbots, and yes, ChatGPT too :) ), LLMs and multimodal models, Causal ML, etc. We…https://t.co/MfnRG6kp7G
For those in India and overseas looking for Indian covid data as numbers begin to rise, a reminder that https://t.co/whh3VBEbNx, our effort at continuing the work of https://t.co/cgHUiqwhah is up and running. Data can be downloaded from the api as before. Corrections welcome.
COVID in India horrifying
Nearly 300,000 infection today, more than 2000 deaths
Great @nytopinion piece on what went wrong
Neglect, poor messaging, worse policy, false belief that India had beaten COVID
Humanitarian disaster unfolding in front of us
https://t.co/2kmihOIqVN