Gabriel Ceicoschi

Software Engineer shipping AI platforms from Europe and running AI workshops in the Netherlands.

[Gabriel Ceicoschi @gceico]

@mitchellh Noticed same issues, cant merge PRs, blank screens. Nice story, went through you article and got a view of the past. In 2008 I was too young to even know what GitHub it, appreciate the retrospective

[Gabriel Ceicoschi @gceico]

@humalikeai Props to Jared, cool project guys. Is this available for use in Europe? I would like to try it out, but not sure if it follows our GDPR rules. Any thoughts on that?

[Gabriel Ceicoschi @gceico]

Happy startup founder. Check this out.

https://t.co/eIouUcpjJ3

[Gabriel Ceicoschi @gceico]

cool

[Andrej Karpathy @karpathy]

Excited to release new repo: nanochat!
(it's among the most unhinged I've written).

Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.

It weighs ~8,000 lines of imo quite clean code to:

- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.

Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.

My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.

Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.

[Gabriel Ceicoschi @gceico]

@kaysorin @OpenAI @DanielLurie @sama Guys, I have to say that this year’s branding is great. Simple. Relatable and super cute.

[Gabriel Ceicoschi @gceico]

Met the Altman.

[Gabriel Ceicoschi @gceico]

Came here for food and merch.

Happy to meet some people and get inspired.

[Gabriel Ceicoschi @gceico]

Been there! Awesome workshop ✌️

[Kent C. Dodds 🏹 @kentcdodds]

What an awesome crew! Had a great time with the Web Auth workshop attendees today for @reactlivenl!

[Gabriel Ceicoschi @gceico]

Meeting @kentcdodds, Kody the Koala and many other amazing people at #ReactLive Conference in Amsterdam.

[Ray Dalio @RayDalio]

Many of you have asked about finding a work-life balance, so I wanted to share some of the principles that have helped me to achieve this. (1/3)

[Pranay Pathole @PPathole]

"What's the meaning of life ... I came to a conclusion that what really matters is trying to understand the right questions to ask & the more that we can increase the scope & scale of human consciousness the better we are able to ask these questions" — @elonmusk

[Coinbase 🛡️ @coinbase]

Ethereum is the second-biggest cryptocurrency by market cap after Bitcoin, but it's more than digital money — it’s the foundation of a $200 billion-plus economy. From DeFi to digital art, understand the forces at work behind the rise of Ethereum. https://t.co/DM8fNYrrpz

[Anthony Pompliano 🌪 @APompliano]

1 bitcoin is going to be enough for financial freedom for most of the global population.

[Beniamin Mincu |🇺🇸/acc @beniaminmincu]

Had a great conversation with @AlexSaundersAU. We outlined the goal of Elrond, why it matters, and why our unique position opens perhaps the most compelling path to wide-spread adoption.

[Gabriel Ceicoschi @gceico]

@madalina_robu ‘e o metafora’ - subsemnatul 2k17

[Logan Paul @LoganPaul]

HUGE iPhone 7 GIVEAWAY! Just RETWEET this tweet & watch this video 😁 ➡️https://t.co/lTfB5lLAXx