E. Tolga Ayan

Data scientist

[SkyPilot @skypilot_org]

Karpathy's Autoresearch is bottlenecked by a single GPU. We removed the bottleneck.

We gave the agent access to our K8s cluster with H100s and H200s and let it provision its own GPUs. Over 8 hours:

• ~910 experiments instead of ~96 sequentially
• Discovered that scaling model width mattered more than all hparam tuning
• Taught itself to exploit heterogenous hardware: use H200s for validation, screen ideas on H100s

Full setup and results: https://t.co/03WnG9zmzO

@karpathy

[GitHub Projects Community @GithubProjects]

|￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣￣|
| Don't Push To Production On Friday |
|＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿|
\ (•◡•) /
\ /
——
| |
|_ |_

[E. Tolga Ayan @etolga_ayan]

@mertcobanov cemil olabilir mi?

[E. Tolga Ayan @etolga_ayan]

@aokocax Sağ alttaki @RoportajAdam olmuş

[E. Tolga Ayan @etolga_ayan]

@emrehasgulec Eskiden çok güzel ahşap satranç takımları satıyordunuz keşke yine getirseniz

[DAIR.AI @dair_ai]

7/ Beyond Human Data for LLMs - an approach for self-training with feedback that substantially reduces dependence on human-generated data; the model-generated data combined with a reward function improves the performance of LLMs on problem-solving tasks.

https://t.co/pnsP90EHJx

[DAIR.AI @dair_ai]

10/ Quip - compresses trained model weights into a lower precision format; combines lattice codebooks with incoherence processing to create 2 bit quantized models; significantly closes the gap between 2 bit quantized LLMs and unquantized 16 bit models.

https://t.co/w1u6JKHkrg

[E. Tolga Ayan @etolga_ayan]

@deci_ai Why did you report results on two different GPUs? Isnt it more beneficial to compare vLLM vs Huggingface vs Infery-LLM?

[Microsoft Research @MSFTResearch]

Today, we share our teams’ latest contributions, Phi-2 and promptbase.

Phi-2 outperforms other existing small language models, yet it’s small enough to run on a laptop or mobile device. https://t.co/wLhUeRsByL

[E. Tolga Ayan @etolga_ayan]

@efsaneyihatrla Sınavdan önceki gün sınav soruları cevapları siye bir link paylaştı. Açınca kedisinin fotosu çıkıyordu.

[E. Tolga Ayan @etolga_ayan]

@efsaneyihatrla Bilkentte bahar festivali mayfest olurdu. O güne sınav koymuştu sınav duyurusunu da mayfest temasıyla süslemişti böyle bir kraldı.

[E. Tolga Ayan @etolga_ayan]

@abhi1thakur they should redirect the flow using a residual block.

[eskicii @rejavnt]

Kanka şimdi şöyle zaten biliyosun bizim muhabbet makara. Kaydedicez bunu. Spotify podcast yapcaz.

[E. Tolga Ayan @etolga_ayan]

@lesavip

[Ben Tossell @bentossell]

All the best examples of ChatGPT, from OpenAI: