Adrian Matei

Data & #AI Director (CDO, CAIO) | Digital Transformation Executive | Cloud & Enterprise IT Director

[Andrej Karpathy @karpathy]

Software horror: litellm PyPI supply chain attack.

Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords.

LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm.

Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery - Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks.

Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages.

Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible.

[Adrian Matei @adrmatei]

https://t.co/KAw5DKdDA9

[Adrian Matei @adrmatei]

AI isn’t a tool anymore, it’s a teammate. But who’s keeping it in check?

🤖 Autonomous agents are acting, deciding, even collaborating.
👥 Humans are still accountable — but no longer fully in control.
💥 Chaos is just one ungoverned prompt away.

Enter the Agentic AI CoE.

[Thomas Kurian @ThomasOrTK]

Google Cloud is committed to open and flexible AI ecosystems that help you build solutions best-suited to your needs. We’re excited to bring Llama 3.1 to Vertex AI Model Garden, including 405B—Meta's most powerful and versatile model to date. https://t.co/iz07hi4rwP

[Steve Jarrett @stevejarrett]

Rich natural interaction with computers is going to change many things, but especially education.

This @OpenAI math tutor demo by @salkhanacademy of Khan Academy with his son is extraordinary. Next we need to figure out how to provide these tools to people in poor communities and developing economies.

[Christel Heydemann @Cheydema]

L’intelligence artificielle est dans toutes les discussions et était naturellement au 🧡 de cette 5ème édition de l’#OBSummit.
➡️ L’adoption de l’#IA par les entreprises va nécessiter davantage de besoins en disponibilité, en performance, en flexibilité et en sécurité des réseaux. Et également beaucoup de confiance, avec des partenaires de choix !
Une journée couronnée par le lancement de notre nouvelle offre d’#IA Générative clé en main.
Un grand bravo @AlietteML et aux équipes @orangebusiness. 👏

[Steve Jarrett @stevejarrett]

CEOs looking to define their AI strategy finally have a dedicated online class to guide them. It features brilliant @coursera CEO Jeff Maggioncalda, AI pioneer @AndrewYNg, @microsoft CEO @satyanadella, and many other experts including a technical strategy section with me. We all hope you find it useful and look forward to your feedback!

https://t.co/YHPLoTt2Pt

#AI #GenAI #LLM #PromptEngineering #FineTuning #AIAct #ResponsibleAI

[julien ducarroz @jducarroz]

Today, I shared my enthusiasm & hopes for 2024 with our newcomers in a lively @orangeromania meeting. Talked about the great people we have, reality & expectations of early days, sharing #LifeAtOrange insights.
Refreshing start, let's begin the journey on a high note together!🚀

[Adrian Matei @adrmatei]

@jducarroz @orange @stevejarrett Great exchange Julien, thanks for the opportunity to share on the potential of Data & AI to fuel business digital transformation! Our upcoming journey is one centered on people empowerment, business drive & citizen development, as we adopt technology to help us be the best we can

[Sam Altman @sama]

the mission continues

[Thomas Kurian @ThomasOrTK]

Today we’re announcing new data residency guarantees so customers using Google Cloud’s Vertex AI platform can store their data at-rest in any of 10 available countries. https://t.co/7bPvv4L0Zg

[Liudmila Climoc @Liudmila_Climoc]

Google Cloud Day in @Orange_Polska. We're sharing best practices and discuss the future possibilities of #data, #cloud & the magic of #AI. Thank you @mno_jl @MagdaDziewgu @stevejarrett @behshad_behzadi & all the speakers for your insights and experiences. Let's scale up together!

[Steve Jarrett @stevejarrett]

OpenAI’s acquisition of Global Illumination may be a sign that multi-model / ensemble architectures that include a physical model of the real world may be necessary to reach AGI.

@ylecun and others have been making compelling arguments for that approach for some time. This news seems like additional affirmation.

https://t.co/5MDlzXhhNz

[Steve Jarrett @stevejarrett]

The open source @Meta #LLaMA #AI will allow us to create much smaller, faster, and more energy efficient yet still powerful models. At @Orange we are committed to responsible AI that gives superpowers to our employees, our customer interactions, and our networks. We are impressed with the open source AI work by Meta, particularly the research team in Paris founded by @ylecun. Excited for our collaboration!

https://t.co/ElaBGr8ieW

[Andrej Karpathy @karpathy]

I think this is mostly right.
- LLMs created a whole new layer of abstraction and profession.
- I've so far called this role "Prompt Engineer" but agree it is misleading. It's not just prompting alone, there's a lot of glue code/infra around it. Maybe "AI Engineer" is ~usable, though it takes something a bit too specific and makes it a bit too broad.
- ML people train algorithms/networks, usually from scratch, usually at lower capability.
- LLM training is becoming sufficently different from ML because of its systems-heavy workloads, and is also splitting off into a new kind of role, focused on very large scale training of transformers on supercomputers.
- In numbers, there's probably going to be significantly more AI Engineers than there are ML engineers / LLM engineers.
- One can be quite successful in this role without ever training anything.
- I don't fully follow the Software 1.0/2.0 framing. Software 3.0 (imo ~prompting LLMs) is amusing because prompts are human-designed "code", but in English, and interpreted by an LLM (itself now a Software 2.0 artifact). AI Engineers simultaneously program in all 3 paradigms. It's a bit 😵‍💫

[Thomas Kurian @ThomasOrTK]

We are excited about our partnership with @Quora and making our PaLM 2 model available on @poe_platform.

[Steve Jarrett @stevejarrett]

At @orange we are proud to have chosen @googlecloud as our AI and data cloud partner. In particular, we have a strong culture fit with their core engineering teams and work together closely as co-design partners on a number of projects.

I’m particularly excited to show off in August at Google Cloud Next what we have been working on with the brilliant and fun pair of @behshad_behzadi and Ankur Jain. It was great to have both of you guys in Paris!

Thanks also to Google Cloud CEO @ThomasOrTK for coming to France and taking time to listen to our needs and ideas.

[Adrian Matei @adrmatei]

@SurugiuOlga It was great to discover an amazing community in Moldova & share on how we use tech to drive digital transformation! Really impressed with the level of maturity of its adoption, excited to build more projects together! Thank you @SurugiuOlga for the invite & being the best host!

[Steve Jarrett @stevejarrett]

Impressive progress by @GoogleAI in computer understanding of graphs with extraction of the underlying data as well as answering math problems written in plan language.

What’s even more impressive is that they can use another model in an ensemble to generate the Python code to do the same tasks.

These kids of advances and easy to use tools are going to provide mathematics superpowers to people all around the world.

We’ll still need people skilled in math to check for errors on the important questions, though! But again there will be AI tools to help them as well…

Also, expect to see many more advances as many different models are ‘chained’ together in an ensemble and so are able to tackle very increasingly complex tasks.

Exciting times…

[Forbes @Forbes]

OpenAI CEO Sam Altman expressed concern about the European Union’s attempts to regulate artificial intelligence and warned the company may have to pull its services from the region if it is unable to comply with the regulations. https://t.co/vqRKLqxUSl