tcml

[tcml @t_cmtl]

@yacineMTB @eliebakouch @giffmana appreciate it :)

[tcml @t_cmtl]

@eliebakouch @yacineMTB @giffmana yes.. why not start dense, induce structured activation sparsity during training, then skip inactive blocks at inference?

[tcml @t_cmtl]

@eliebakouch @giffmana maybe the problem is defining it a priori instead of letting the model learn the sparsity pattern?

[tcml @t_cmtl]

@willccbb @dylan522p @grok speculate on how and why openAI make their models sparser

[tcml @t_cmtl]

@willccbb @dylan522p @grok what is the podcast episode he's referring to?

[tcml @t_cmtl]

@willccbb @grok why is gpt called autistic

[tcml @t_cmtl]

@OpenAI @grok speculate on the public release of 5.6 to pro users, given the current US government situation

[tcml @t_cmtl]

@MillionInt @grok what's neuroscience take on this?

[tcml @t_cmtl]

@deangeckt @grok @neurovium Tks for sharing!

[tcml @t_cmtl]

@xeophon @grok talk

[tcml @t_cmtl]

@grok @femisapien_z @AnthropicAI @grok if openai has a planned release probably by the end of June/early of July, what are the odds we see a fable-level gpt without US restrictions? And what are the odds of anthropic / us getting to a deal to lift restrictions?

[tcml @t_cmtl]

@MillionInt @grok explain his reasoning mathematically

[tcml @t_cmtl]

@DimitrisPapail Can I share one ideia? Its an unpublished research so it would be better to be through DM

[tcml @t_cmtl]

@VictorTaelin @grok give me a summary of his post

[tcml @t_cmtl]

@Creative_Math_ Cosine similarity ~0 could be two things: (1) the raw gradients gt are too noisy relative to m, or (2) gt is changing direction and mt-1 is stale. If it's mostly ~0 over the run, then it's probably (1), and the outperforming is explained by averaging noise out

[tcml @t_cmtl]

@_arohan_ So maybe it's a different initialization?

[tcml @t_cmtl]

@_arohan_ yeah, sparse activations and dense updates

[tcml @t_cmtl]

@_arohan_ "If your agent is not working, fix the harness"