@karpathy Test Time Training-style approaches feel promising here: replace or augment the KV cache with a small learned model that gets weight updates in-context.
@ID_AA_Carmack I have spent a lot of time thinking about how to apply RL and imitation learning to robotics. Solving the Atari games this way interestingly forces you to consider many of the challenges faced in robotics that we often sidestep when solving atari via pure software emulation.
@KaspaFacts According to this:
https://t.co/xFwnG2yP62
The fee is about 0.73 kas which is much more than 1/3000 of a penny. Can you or someone explain the discrepancy ?
@ylecun I was at ICLR 2013 in Scottsdale, AZ. I remember it seemed like a much smaller conference compared to others. Everyone was talking about dropout, and Hinton joined remotely over the banquet dinner to talk about DREDNETS (deep relu networks with dropout).
@DisplacedGamers This is actually very easy to fix. Same for the Mario 3 pause screen. It's just the nmi interrupt handler (which fires at the beginning of the scanline) not being fast enough.
@karpathy I remember this! iirc, Krizhevsky originally wrote this as an assignment for a gpu programming class at stanford? But then it became a thing of its own.
@elonmusk I never thought I'd say this but I have been consistently getting better search results on bing. As cool as it is, I don't think this AI will save them. Hopefully they step up to the challenges.
@gfodor@ID_AA_Carmack can you tell me what book or author? every time I try different search terms for books "hamming distance" always comes up instead of an author name
@ID_AA_Carmack Like histogram equalization? probably no benefit. Models perform well at transforming between distributions. VAEs reshape gaussians for example. Scale and bias are harder to learn since the range of the initial model weights determine the input scale the model can work with
@fchollet While I see your point, I feel this is just the next step in moving the bar for what we consider AGI. Now even if we have an intelligent agents that can write poems, stories and even create amazing art, its still not AGI because its output must be observed by a human?
@jeremyphoward@jan_czechowski@BigScienceLLM I have seen this. It occurs if the spike is due to a large gradient causing large weights, which further causes some activations to have large negative values which can get stuck. RELU functions output 0 for negative numbers which also produces no gradient, not allowing recovery.
@bittrexexchange Please help me verify my account. My funds are locked. I followed instructions and got "name mismatch" No help from open support ticket. I'm "[email protected]"