Safiuddin Khaja

Data Enthusiast

[Safiuddin Khaja @safi1409_]

@kaleybrauer Does this mean I can get high thinking while keeping the thinking level as low by adding more dots?

[Xiao Ma @infoxiao]

I just open sourced my “lazy or craftsman?” simple test.

[Safiuddin Khaja @safi1409_]

@GergelyOrosz Idx is firebase studio.

[Safiuddin Khaja @safi1409_]

@VTchuiev @iruletheworldmo Gemini 2.5 still leads Simple bench. I guess Google has some training data similar to it.

[Safiuddin Khaja @safi1409_]

My father-in-law is an ML engineer. He’s insanely gifted.
We were looking at an old model checkpoint that had been trained with SFT years ago, and I asked him what it would take to reproduce it today.
I’ll never forget his answer.

“We can’t do it. We don’t know how.”

[Lucas Beyer (bl16) @giffmana]

Wow!! Google discovering AND OPEN-SOURCING the latest training techniques such as supervised finetuning (SFT) wasn't on my bingo card.

Soon they will have caught up with the frontier, and are sharing this with all of us!

[Safiuddin Khaja @safi1409_]

@giffmana Google has moat. It's called SFT

[Sauers @Sauers_]

After training their flagship 405B parameter model, Thinking Machines researchers discovered that replacing identity mappings between attention layers with non-linear activation functions dramatically improved performance. "Our previous architecture was essentially computing weighted averages at every layer," explains lead researcher; "introducing non-linearity allows the network to learn feature interactions we didn't know were possible—it can now represent functions that aren't just linear combinations of inputs." The lab is calling this the "Deep Learning 2.0" paradigm shift.

[Safiuddin Khaja @safi1409_]

@giffmana This is weird to me. I use gemini with 600-700k context (mostly complete coderepos) and it works pretty well. Granted I don't use the CLI but do it manually using aistudio.

[Safiuddin Khaja @safi1409_]

@kosa12matyas More like Shazeer's benevolence

[Safiuddin Khaja @safi1409_]

@IceSolst Please share the chrome extension link. Asking for a friend.

[Safiuddin Khaja @safi1409_]

@aidan_mclau @willccbb And if you pass max_output_tokens as zero, that isn't G, it's just

[Safiuddin Khaja @safi1409_]

@rooftop_labs @0xIlyy Just do rm -rf .git

[Safiuddin Khaja @safi1409_]

Not necessarily unless your new benchmark is as "wide-ranging" as the LMSys queries.

[James Campbell @jam3scampbell]

the fun thing about designing unconventional benchmarks is that you can instantly see which models were desperately overfit to LMSYS to please managers vs which were focused on raw intelligence (hint, the new 3.5 sonnet)

[Safiuddin Khaja @safi1409_]

@paul_cal @doomie An employee who did not want to be named revealed another scoop: they're allegedly training models on TPUs—Transformer Processing Units. Apparently, this is to keep the flux capacitors in sync with reverse entropy gradients.

[Safiuddin Khaja @safi1409_]

@infoxiao I thought it was deprecated in favour of HP. Much cleaner.

[Safiuddin Khaja @safi1409_]

@vikhyatk Even if you train you can't upload it to huggingface.

[Safiuddin Khaja @safi1409_]

@aidan_mclau Gemini Pro 001 or 002. Also where is Flash?

[Safiuddin Khaja @safi1409_]

@yar_vol @simonw @__p_i_o_t_r__ Generally more parmas leads to better creativity I've seen. For some reason smaller models even though have become very smart are not even close in creativity.

[Safiuddin Khaja @safi1409_]

@mervenoyann Gemini Flash answers correctly