Magnitude-preserving Discriminators Enable Efficient GANs

NeurIPS 2026

Nick Yiwen Huang, Jackson Woodleigh∗, Aaron Gokaslan∗†, Xinjie Yi‡, James Tompkin

Brown University   †Modal   ‡Independent   ∗Equal contribution

Efficiency without tricks

190× less compute than guided diffusion at the same FID

5–10× fewer parameters than guided one-step flow models at the same FID

Samples

Class-conditional ImageNet 256×256.

Guidance trades diversity for fidelity: it pushes each sample toward the most typical images of its class, which FID can reward. An efficient generative model should learn the underlying distribution without that help.

Speed

Ideal time per image at a sustained 20 TFLOP/s, from each method’s compute, with every latent model using a tiny distilled decoder (TAESD , TAEF2  or the like). FIDs are with each method’s full decoder.

GANdscape

R3GAN2 is fast enough to fill a zoomable map of all 1,000 ImageNet classes live in your browser.

Open the ImageNet GANdscape

Generator

Pick a class and walk through R3GAN2’s latent space, generated on your own GPU in the browser.

Abstract

GANs are appealing because they generate sharp images with a single forward execution. Recent works have stabilized GAN training, but FID may still plateau on diverse datasets like ImageNet-256 even as training remains stable. We identify severe growth of discriminator activation magnitudes as a hidden pathology accompanying this plateau. Our analysis shows that adversarial training permits the discriminator to increase empirical real–fake separation by amplifying already-discriminative responses, without necessarily improving the underlying representation. Motivated by this observation, we redesign R3GAN’s network components to preserve activation magnitudes statistically. Across a roadmap of configurations, we perform diagnostic ablations showing that the gains depend primarily on the discriminator redesign. We train all generators and discriminators from scratch, without pretrained discriminator features. The resulting R3GAN2 models reach FIDs of 2.12 with 17 M parameters, 1.60 with 61 M, and 1.52 with 131 M on class-conditional ImageNet-256. At 1 NFE, this is substantially more efficient than competing diffusion and autoregressive methods, and than prior GAN baselines too (e.g., 10× fewer parameters than StyleGAN-XL for similar FID). We also train directly in pixel space, achieving FIDs of 1.65 on CIFAR-10 and 2.19 on FFHQ 256×256, where our generator achieves comparable FID to StyleGAN-XL with fewer than half as many parameters. In sum, identifying and removing this discriminator magnitude pathology yields a new SOTA in FID/parameter efficiency for GANs.

Citation

@inproceedings{huang2026magnitude,
    title     = {Magnitude-preserving Discriminators Enable Efficient {GAN}s},
    author    = {Huang, Nick Yiwen and Woodleigh, Jackson and Gokaslan, Aaron and Yi, Xinjie and Tompkin, James},
    booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
    year      = {2026}
}

Acknowledgements

NSF CAREER-2144956 and Google’s TPU Research Cloud.

The live generators decode with TAEF2 (Ollin Boer Bohan, MIT licence) and the FLUX.2 autoencoder (Black Forest Labs). Our weights: ImageNet models CC BY-NC 4.0, FFHQ model CC BY-NC-SA 4.0.

References