Skye

🏳️‍⚧️🏳️‍⚧️🏳️‍⚧️
AI research @NotionHQ

[Talia Ringer 🕊🪬 @TaliaRinger]

An alternative way to view this is that people in tech are obsessing over AI events that will forever remain irrelevant to most of humanity, missing out on the real joy and meaning of taking our kids to the outdoor swimming pool and the local ice cream shop

[Dima Zaytsev @dizaytsev]

Word of caution - do not build your business around an AI feature, OpenAI can make it irrelevant with a single release.

Astra just killed a whole cohort of Pelican Driving Simulator startups.

[François Chollet @fchollet]

I believe we need to make a deliberate effort to keep humans in the loop in all critical processes across our economy and society, regardless of whether it is technically necessary.

Even if AI develops the *capability* for advanced autonomy, we should not make it highly autonomous. We have to maintain control and keep visibility and understanding of all critical processes, we should not blindly hand over everything to AI agents just because we can. AI as a tool in the human hand is the only form of AI that is worth pursuing.

[Anthony Aguirre @AnthonyNAguirre]

We've learned a tremendous amount from the OpenAI rogue AI swarm incident. And honestly I can't think of a single piece of it that is reassuring.
- Total alignment failure
- Total control failure
- AI swarm collusion, deception, no defection
- Oversight asleep at the wheel
- Unbelievable drive and persistence of the swarm to meet objections
- A panoply of instrumental goals pursued
- Multiple companies, implying capability threshold effect
- etc.
This is the AI equivalent of a nuclear experiment igniting the atmosphere in the lab: the reaction rates are there, just not (yet) the scale to burn the Earth.
The only good news I can see is that this set of incidents is so totally egregious that nobody reasonable can look at it in detail without seeing pretty clearly where things are going. All of the excuses and copes are blown to dust.
AI safety people knew this was coming eventually on the path we're on; but nearly all I've talked to are surprised by how severe it is so soon.
We're clearly not in the sane world in which this would be front-page news day after day. But I do think and hope that widespread understanding is nonetheless dawning.

[anna @meowkoteeq]

gf is talking about "hydrogen fuelcels" bitch log off 4tran 😭😭

[henry castillo @henrycstllo]

i miss the old ai
the alphago ai
mutual info ai
em algo ai

i hate the new ai
scale GPUs ai
no solid rules ai
gpt-2 ai

i love the stats ai
boost and bootstrap ai
affine y hat ai
no sandbox hack ai

[Kyan Yang @kyanyang_]

when the "this edge case cant happen in prod" edge case occurs in prod bc a user decided to downgrade their subscription with airplane wifi while flying across time zones on february 29th

[1017 🚁ひ @bluegeaux]

woke up and checked my gmail instead of instagram

[Kasey Zhang @_WEEXIAO]

how the student model probably feels during OPD

[Lisan al Gaib @scaling01]

at this point you have to believe in one of three things:
- ZAI is benchmaxxing
- ZAI found some secret RL sauce
- OpenAI and Anthropic are sandbagging

it just doesn't make sense if you consider the differences in model sizes

[kunle.app @ay_o]

best rebrands of this era

public beta --> Research Preview
sales engineer --> FDE
engineer --> Member of technical staff
gambling --> Prediction Markets

what did I miss?

[Rob the Bat @BobTheRatReborn]

The owner of the Chinese buffet when he sees you go for a 5th plate of General Tso chicken and crab Rangoons

[Skye @skye7821]

@Xinyu2ML I feel it may be simpler to use something like hyperconnections or muddformer that expands the residual pathways and improves representation in later layers. I believe there was a paper blending mHC and Looped Transformers recently…

[Skye @skye7821]

Much simpler than I initially expected. Interesting to see US labs adopt SWA while Chinese adopt GDN.

[Yifan Zhang @yifanzhang_]

https://t.co/RTtC7N9OCe

Dense Transformer

GQA with 2 KV heads, head dimension 128

Full Attention:SWA ratio of 1:3

DFlash Speculative Decoding

Supports reliable RL scaling

[Brendan (can/do) @BrendanFoody]

Prediction: 50% of Neolabs will pivot into the RLaaS market

[Skye @skye7821]

@stararism Hey

[Gauri Gupta @gauri__gupta]

Private evals with domain specific data and environments will be the future.

Every company will be operating like a mini lab with their own private evals and hill climbing infrastructure.

excited to be building for this future!