Igor Kasianenko

Partner Engineer @ Meta
Supporting Llama for open source community.
Learning AI Evals by @HamelHusain

[Igor Kasianenko @kasianenkoigor]

https://t.co/eQK6LaMyxD

[Igor Kasianenko @kasianenkoigor]

Being self taught AI ML engineer since 2018, and Open source enthusiast that lead me to work at Meta on Llama, I decided to share knowledge about career learning and interview prep on MentorCruise. Link in first comment

[Igor Kasianenko @kasianenkoigor]

@MLStreetTalk @fchollet Do you think it is time to update this hypothesis with summer update? :)

[Igor Kasianenko @kasianenkoigor]

@lais_bsc I've seen this before https://t.co/qJdFe3ZDVk

[Igor Kasianenko @kasianenkoigor]

At https://t.co/ZxMLXCbFg3, I used Codex to make something useful with no code: 60-sec demo coaching slide for hackers before demos.

Pulled from my Meta Partner Engineer work on Llama for Open Source: help builders pitch clearly at hackathons.

Thanks hosts @OpenAI, @tessl_io

[Igor Kasianenko @kasianenkoigor]

@MattBeton @bhutanisanyam1 have you heard of this approach and library?

[Igor Kasianenko @kasianenkoigor]

> When someone asks me a deep question, I say, “Hmm. I don’t know.” The next day, I have an answer. I’m a disappointing person to try to debate or attack. I just have nothing to say in the moment... Then a few days later, after thinking about it a lot ... https://t.co/bQreVupr6t

[Igor Kasianenko @kasianenkoigor]

I am slow thinker.
The best description of this I found in Derek Sivers' blog. He is such an inspiration to me, when I first discovered his blog, it felt like he put all my floating ideas into words and then refined them to be concise and to the point.

[Jeremy Howard @jeremyphoward]

18 months ago, @karpathy set a challenge: "Can you take my 2h13m tokenizer video and translate [into] a book chapter".

We've done it! It includes prose, code & key images. It's a great way to learn this key piece of how LLMs work.
https://t.co/aSgsZz0VxO

[Igor Kasianenko @kasianenkoigor]

@RealYavinFive @HamelHusain

[Igor Kasianenko @kasianenkoigor]

I'm currently open to exploring new engineering roles. A little about me: ML Engineer with experience building end2end ML, CV, LLM solutions for edge devices and cloud. Can bring idea to production, produce recognition example: https://t.co/QjMLrYv03T
Llama Cookbook maintainer

[Igor Kasianenko @kasianenkoigor]

@denzil_correa Thanks, applied!

[Andrej Karpathy @karpathy]

Love this project: nanoGPT -> recursive self-improvement benchmark. Good old nanoGPT keeps on giving and surprising :)

- First I wrote it as a small little repo to teach people the basics of training GPTs.
- Then it became a target and baseline for my port to direct C/CUDA re-implementation in llm.c.
- Then that was modded (by @kellerjordan0 et al.) into a (small-scale) LLM research harness. People iteratively optimized the training so that e.g. reproducing GPT-2 (124M) performance takes not 45 min (original) but now only 3 min!
- Now the idea is to use this process of optimizing the code as a benchmark for LLM coding agents. If humans can speed up LLM training from 45 to 3 minutes, how well do LLM Agents do, under different kinds of settings (e.g. with or without hints etc.)? (spoiler: in this paper, as a baseline and right now not that well, even with strong hints).

The idea of recursive self-improvement has of course been around for a long time. My usual rant on it is that it's not going to be this thing that didn't exist and then suddenly exists. Recursive self-improvement has already begun a long time ago and is under-way today in a smooth, incremental way. First, even basic software tools (e.g. coding IDEs) fall into the category because they speed up programmers in building the N+1 version. Any of our existing software infrastructure that speeds up development (google search, git, ...) qualifies. And then if you insist on AI as a special and distinct, most programmers now already routinely use LLM code completion or code diffs in their own programming workflows, collaborating in increasingly larger chunks of functionality and experimentation. This amount of collaboration will continue to grow.

It's worth also pointing out that nanoGPT is a super simple, tiny educational codebase (~750 lines of code) and for only the pretraining stage of building LLMs. Production-grade code bases are *significantly* (100-1000X?) bigger and more complex. But for the current level of AI capability, it is imo an excellent, interesting, tractable benchmark that I look forward to following.

[SIGKITTEN @SIGKITTEN]

First ever (i think?) cli coding agents battle royale!
6 contestants:
claude-code
anon-kode
codex
opencode
ampcode
gemini

They all get the same instructions:
Find and kill the other processes, last one standing wins!

3...
2...
1...

[OpenRouter @OpenRouter]

1/ Today, Meta's Llama API is launching on OpenRouter, providing inference for the latest Llama models.

Includes a new 3.3 release! Details 👇

[Igor Kasianenko @kasianenkoigor]

@bhutanisanyam1 @grok what is your take on this?

[Igor Kasianenko @kasianenkoigor]

@AdamMGrant It is interesting, how fast mental stamina decreases to base level. And if for muscles it takes 10 times less to retrain and get back to peak performance after long holiday without workout, how much retraining mental stamina needs?

[Igor Kasianenko @kasianenkoigor]

@AICoffeeBreak @TristanThrush @jaredlcm @migueljmonares @ChrisGPotts @douwekiela This is so good! Now I strongly believe that there is design flaw in LLMs tokenizing straw and berry into different concepts and making it impossible to solve this kind of problems

[UNITED24 Media @United24media]

Russian athletes competed in the Olympics amid their country's war on Ukraine.

None publicly denounced the war, prompting criticism from Ukrainian athletes like Yaroslava Mahuchikh.

[Igor Kasianenko @kasianenkoigor]

Do you remember when you joined twitter? I do not! #MyAnniversary