Zilan

[Evin Kürdü @ulansenkimsinnn]

qûna te rabûye

[xezal @dogacesnicisii]

ulusların kendi şampiyonluğunu tayin hakkı

[jessu @jessu_js]

Prompt mühendisliği 101

[༺☆𝙤̈𝙮𝙠𝙪̈ 𝙙𝙚𝙣𝙞𝙯☆༻ @0ykudeniz]

beyaz yakalıların bowllar

[𝘚𝘦𝘳 𝘌𝘭𝘭𝘪𝘦 ⋆.𖥔⚔️ @gildfae]

The autistic urge to rewatch the LOTR trilogy for the 100th time

[Zilan @llimonataa]

body roll yapmak ve yapamamak butun mesele bu

[Warner Bros. @warnerbros]

We’ve been waiting for you, precious. The Lord of the Rings: The Hunt for Gollum only in theaters 17 December 2027.

[Zilan @llimonataa]

aptal bekir

[Denge+ @dengeplus]

Oyuncu Ufuk Bayraktar, İstanbul'da yapılan feminist yürüyüşüne tepki gösterdi.

Kolluk kuvvetleri oyuncuyu sakinleştirmeye çalıştı. (BirGün)

[K Ü B 🐾 (melek_spor) @pilavsevenpanda]

Yeah sex is cool but have you ever heard “jin jiyancılar tarafından itilerek alandan çıkartılmışlar”

[Erdal Bektas @eeebektas]

Claude muhteşem bir rapor yayınlamış.

Raporun tamamını okumanıza falan gerek yok; diyor ki yapay zekanın birçok sektörde teorik olarak yapabilecekleri çok daha büyük ama henüz ilgili araçlar yok; yapamıyor.

Siz ne yapacaksınız, biliyor musunuz? Bu alanlardan herhangi birini seçeceksiniz ve bu açığı kapatacak uygulamalar hayal edeceksiniz.

[Zilan @llimonataa]

askim hemen taste of cherry ve my favorite cake izliyorsun optm grsrz :*

[Riyad @lahmelevarka13]

Aynen öyle depremzedelerin deprem dönemi de çok sesiydin yardım gelmemesinden doğan öfkeyi devlete yöneltmek yerine Suriyelilere yönelttin kaç tane Antakyalı anadili Arapçayı konuştuğu için yağmacı zannedilip linç edildi senin yüzünden

[Zilan @llimonataa]

Bilet fiyatlarını geçtim de konseri 10 ekime koymak kimin fikriydi

[Biletix @Biletix]

Türkiye alternatif rock sahnesinin en özgün ve öncü gruplarından Çilekeş, Y.O.K albümlerinin 21. yılında sahnelere geri dönüyor!! 📢📢

Görkem Karabudak, Ali Güçlü Şimşek ve Cumhur Avcil'den oluşan orijinal kadrosuyla yeniden bir araya gelen Çilekeş, Epifoni organizasyonuyla 10 Ekim gecesi İstanbul'da KüçükÇiftlik Park'ta sahne alacak.😍

Biletler 28 Ocak saat 12:00'de satışta! https://t.co/GXf11CZeze

[Zilan @llimonataa]

ankara echoes istanbul konser biletlerini yediniz mi arkdşlr

[Andrej Karpathy @karpathy]

New post: nanochat miniseries v1

The correct way to think about LLMs is that you are not optimizing for a single specific model but for a family models controlled by a single dial (the compute you wish to spend) to achieve monotonically better results. This allows you to do careful science of scaling laws and ultimately this is what gives you the confidence that when you pay for "the big run", the extrapolation will work and your money will be well spent. For the first public release of nanochat my focus was on end-to-end pipeline that runs the whole LLM pipeline with all of its stages. Now after YOLOing a few runs earlier, I'm coming back around to flesh out some of the parts that I sped through, starting of course with pretraining, which is both computationally heavy and critical as the foundation of intelligence and knowledge in these models.

After locally tuning some of the hyperparameters, I swept out a number of models fixing the FLOPs budget. (For every FLOPs target you can train a small model a long time, or a big model for a short time.) It turns out that nanochat obeys very nice scaling laws, basically reproducing the Chinchilla paper plots:

Which is just a baby version of this plot from Chinchilla:
Very importantly and encouragingly, the exponent on N (parameters) and D (tokens) is equal at ~=0.5, so just like Chinchilla we get a single (compute-independent) constant that relates the model size to token training horizons. In Chinchilla, this was measured to be 20. In nanochat it seems to be 8!

Once we can train compute optimal models, I swept out a miniseries from d10 to d20, which are nanochat sizes that can do 2**19 ~= 0.5M batch sizes on 8XH100 node without gradient accumulation. We get pretty, non-itersecting training plots for each model size.

Then the fun part is relating this miniseries v1 to the GPT-2 and GPT-3 miniseries so that we know we're on the right track. Validation loss has many issues and is not comparable, so instead I use the CORE score (from DCLM paper). I calculated it for GPT-2 and estimated it for GPT-3, which allows us to finally put nanochat nicely and on the same scale:
The total cost of this miniseries is only ~$100 (~4 hours on 8XH100). These experiments give us confidence that everything is working fairly nicely and that if we pay more (turn the dial), we get increasingly better models.

TLDR: we can train compute optimal miniseries and relate them to GPT-2/3 via objective CORE scores, but further improvements are desirable and needed. E.g., matching GPT-2 currently needs ~$500, but imo should be possible to do <$100 with more work.

Full post with a lot more detail is here:
https://t.co/na8zVLqWLf
And all of the tuning and code is pushed to master and people can reproduce these with scaling_laws .sh and miniseries .sh bash scripts.

[Zilan @llimonataa]

ankara ofis çok kötü iyykk öğkk yani

[ꪆৎ @fairiehaze]

flower embroidered jeans 👖🌻