Gurvan

I like Super Smash Bros. Melee for the Nintendo Gamecube™, Machine Learning, and most of all taking care of my friends.

[Gurvan @gurvanson]

@jsuarez what do you think about MinGRU for RL ?

[Gurvan @gurvanson]

@yacineMTB thank you

[Gurvan @gurvanson]

@jeremyphoward @ggerganov I understand that the amount of memory is a bottleneck on consumer GPUs, but wouldn't the inference speed still be better with less active parameters during generation?

[Gurvan @gurvanson]

@kalomaze if the issue is stable ssh try mosh maybe?

[Gurvan @gurvanson]

@dearmadisonblue is there a normal range of behavior for an egregore?

[Gurvan @gurvanson]

@SSBM_Arte I think it happened to me once when I had paste an image that wouldn't properly upload, but refreshing the page fixed it iirc. It could also be that a previous message is open for editing i guess

[Gurvan @gurvanson]

@kalomaze you should check phillip if you haven't https://t.co/n2CvMAOMry

[Gurvan @gurvanson]

@qtnx_ talking about it tricks you into thinking you've already done something. don't fall for it

[Gurvan @gurvanson]

@jaxmorphy i mean, 3 denoising steps is not a lot. you can still see the stage outline really well. do you plan on using rolling diffusion/diffusion forcing?

[Gurvan @gurvanson]

@jaxmorphy that's the world model i want to see

[Gurvan @gurvanson]

@y0b1byte i think the DreamerV3 paper mentions that it uses the same set of hyperparameters for every experiment, so the comparison might not be entirely fair

[Gurvan @gurvanson]

@iScienceLuvr already done by Schmidhuber in Recurrent Highway Networks https://t.co/3d27pWYCXD
better illustrated here https://t.co/FAQNIKBG5W

[Gurvan @gurvanson]

@spikedoanz @filipviz normalize by average won't work with negative logits. you could offset everything by the smallest logit maybe, and it would also get you the translation invariance property of softmax

[Gurvan @gurvanson]

@spikedoanz still not idempotent tho

[Gurvan @gurvanson]

@rami_mmo that's good to know, thank you for answering!

[Gurvan @gurvanson]

@rami_mmo this seems contrary to what's commonly though about quantized latents, and about why VQVAE was made in the first place. do you think KL works better here because the minecraft scenery is not that diverse? (i think you allude to this in the article)

[Gurvan @gurvanson]

@torchcompiled @giffmana I think the issue here is that for Transformer they plot the cumulative training time (124M+354M+757M+1.4B), instead of comparing to just the 1.4B trained from scratch, which seems to take about the same amount of TPU hours as the Tokenformer 1.4B, so the graph seems disingenuous