zonkedNGray

Like a poorly assembled electrical ball

[zonkedNGray @zonkedNGray]

@LodestoneRock or worse, the model overfits on low noise levels and this overfit corrupts the overall discriminator representation and makes it ill-behaved on high noise levels as well

[zonkedNGray @zonkedNGray]

@LodestoneRock If it’s instance noise with an adaptive strength, I’m curious how well it works beyond mnist. A few months ago I tried adaptive diffusion style instance noise hoping that it can replace domain specific data augmentions and it didn’t work very well.

[zonkedNGray @zonkedNGray]

@LodestoneRock For a strong enough discriminator, either the noise is too weak and D still overfits or the noise destroys all meaningful information and the models learns to ignore the higher noise levels

[zonkedNGray @zonkedNGray]

@wavefunk_ @dvsch We never took the effort to investigate why stylegan failed so miserably on imagenet and what we introduced in r3gan made it work, loss? architecture? both? It’d be a nice result to include in the paper if we had more time

[zonkedNGray @zonkedNGray]

@wavefunk_ @dvsch Also stylegan doesn’t work on imagenet, with or without augmentation, the result is extremely bad. r3gan works out of the box on imagenet and this is important because it’s an initial step towards *general* image generation with GANs.

[zonkedNGray @zonkedNGray]

@wavefunk_ @dvsch We didn’t use ADA, we used non-leaky augmentation with none of the adaptive tricks. Augmentation is a standard practice for training neural nets across different tasks and problem settings, and it’s not GAN-specific.

[zonkedNGray @zonkedNGray]

@wavefunk_ @dvsch For unimodal datasets like ffhq, it’s likely that you won’t get huge improvements since stylegan already does a reasonably good job on these. Though I’d say the improvement compared to stylegan2 is noticeable if you pay close attention to things like hair texture.

[zonkedNGray @zonkedNGray]

@jxmnop Generative models in general (GAN, diffusion, VAE, all included) don’t work well with normalization layers that operate on activation maps. It’s only beneficial to apply normalization in the weight space (weight norm, spectral norm, etc.)

[zonkedNGray @zonkedNGray]

@jxmnop I would recommend that DO NOT use normalization layers in GANs, especially batch norm, it’s extremely harmful to the training dynamics. It’s one of the points we discussed in the R3GAN paper.

[zonkedNGray @zonkedNGray]

@mgostIH RpGAN doesn’t work well without gradient penalties, did you implement R1 and R2 as in the paper?

[zonkedNGray @zonkedNGray]

@SwayStar123 https://t.co/z3sTQPgKjJ

[zonkedNGray @zonkedNGray]

@SwayStar123 It seems you implemented the saturating GAN loss, this is the loss that most GAN analyses (including our paper) focus on because it’s easier to analyze. In practice, we use the non-saturating loss because it has better gradient behavior and regularization effects

[zonkedNGray @zonkedNGray]

Where do I go?

[zonkedNGray @zonkedNGray]

_trace

[zonkedNGray @zonkedNGray]

untitled

[zonkedNGray @zonkedNGray]

fold

[zonkedNGray @zonkedNGray]

the strangest summer

[zonkedNGray @zonkedNGray]

...woozy anticipation

[zonkedNGray @zonkedNGray]

_orifice

[zonkedNGray @zonkedNGray]

vague yet