semih duru

[semih duru @semihduru]

@OctopusEnergy there are bills from 2023 appeared on my account which i paid already that time. What is happening with your systems? It is impossible now to track and understand them ☹️

[Jason Weston @jaseweston]

🚨New paper!🚨
Self-Rewarding LMs
- LM itself provides its own rewards on own generations via LLM-as-a-Judge during Iterative DPO
- Reward modeling ability improves during training rather than staying fixed
...opens the door to superhuman feedback?
https://t.co/YZRuFphh0H
🧵(1/5)

[Hyung Won Chung @hwchung27]

I gave a talk at Seoul National University.

I titled the talk “Large Language Models (in 2023)”. This was an ambitious attempt to summarize our exploding field.

Video: https://t.co/vumzAtUvBl
Slides: https://t.co/IidLe4JfrC

Trying to summarize the field forced me to think about what really matters in the field. While scaling undeniably stands out, its far-reaching implications are more nuanced. I share my thoughts on scaling from three angles:

1) Change in perspective is necessary because some abilities only emerge at a certain scale. Even if some abilities don’t work with the current generation LLMs, we should not claim that it doesn’t work. Rather, we should think it doesn’t work yet. Once larger models are available many conclusions change.

This also means that some conclusions from the past are invalidated and we need to constantly unlearn intuitions built on top of such ideas.

2) From first-principles, scaling up the Transformer amounts to efficiently doing matrix multiplications with many, many machines. I see many researchers in the field of LLM who are not familiar with how scaling is actually done. This section is targeted for technical audiences who want to understand what it means to train large models.

3) I talk about what we should think about for further scaling (think 10000x GPT-4 scale). To me scaling isn’t just doing the same thing with more machines. It entails finding the inductive bias that is the bottleneck in further scaling.

I believe that the maximum likelihood objective function is the bottleneck in achieving the scale of 10000x GPT-4 level. Learning the objective function with an expressive neural net is the next paradigm that is a lot more scalable. With the compute cost going down exponentially, scalable methods eventually win. Don’t compete with that.

In all of these sections, I strive to describe everything from first-principles. In an extremely fast moving field like LLM, no one can keep up. I believe that understanding the core ideas by deriving from first-principles is the only scalable approach.

[semih duru @semihduru]

@imTolstoyevski Neden olmasin ? https://t.co/eqEWgBoQ5w

[Lex Fridman @lexfridman]

GPT-3 has 175 billion parameters (synapses). Human brain has 100+ trillion synapses. In 2020, it costs ~$2.6 billion to train neural net the size of the human brain. If ML training efficiency improvement trends continue, in 2040, it'll cost $80,000. Video: https://t.co/pVKbPxnbRy

[faraaz @faraaz]

I got GPT-3 to start writing my SQL queries for me

p.s. these work against my *actual* database!

[hasan söylemez @hasansoylemez]

İnsan kendi içine doğru bir yolculuk yapmak istiyorsa fiziksel olarak da bir yolculuk yapmalı. En zoru, insanın kendisiyle yüzleşmesidir. Yalnız ve uzun yolculuklarda bunu yaşamak kaçınılmazdır. Bazen kendinizden nefret edersiniz. Geçmişle, hayatla, insanlarla hesaplaşırsınız.

[Andrej Karpathy @karpathy]

new blog post: "Biohacking Lite" https://t.co/MBr2S88RUv fun to write / learn about quick tour of human energy metabolism

[Eric Berger @SciGuySpace]

Finally, it feels like the 21st century.

[Tim Rocktäschel @_rockt]

I am proud to announce the release of the NetHack Learning Environment (NLE)! NetHack is an extremely difficult procedurally-generated grid-world dungeon-crawl game that strikes a great balance between complexity and speed for single-agent reinforcement learning research. 1/

[Greta Thunberg @GretaThunberg]

Hang in there! Help is available.

[The Economist @TheEconomist]

"Susamam" ("I can't stay quiet") features a parade of 20 rappers calling out Turkey's injustices, crime and corruption https://t.co/zrntxoHsVo

[Ece Temelkuran @ETemelkuran]

Important note for European countries and the US: Turkey is so far the first country that defeated the new global wave of authoritarianism through democratic means- and after 17 years of such an oppresive regime. Hopefully this experince will be studied by the global opposition.

[Fırat Tanış @firattanis]

o ceketi çıkarttırmayacaktınız

[Barış Yarkadaş @barisyarkadas]

ŞU AN...

[Ece Temelkuran @ETemelkuran]

Follow Turkey-Istanbul today to see what it takes to recover a lost democracy. Millions will be defending their right to be political subjects without any expectation from state institutions or any guarantee of security. May the human dignity win today in my city, Istanbul.

[Linus Tech Tips @LinusTech]

Make Mac Pro Grate Again

[#Svarog© 🇹🇷 @Svarogovic]

İmamoğlu bir KIVILCIM olup Tayyip'in yıllardır kurduğu KORKU duvarını yakıp yıktı!

Korkmuyoruz, susmayacağız! #konuşmakgüzel

[John McAfee @officialmcafee]

Her şey çok güzel olacak. #HerşeyÇokGüzelOIacak