论文笔记：unsupervised representation learning with deep convolutional generative adversarial networks

1. previous work [generative adversarial nets]

paper link: http://arxiv.org/pdf/1406.2661v1.pdf 

torch implementation: https://github.com/soumith/dcgan.torch 

a. motivation: supervised network requires huge labeled data ==> generative net generates samples and helps to build unsupervised models based on CNN (Markov Chain no longer needed)  ==> Adversarial nets (GANs) 

b. architecture:



Generative net (G(z)):

input Z: noise vector; output: fake image G(z) and be fed to Discriminator D(x) 

Discriminative net D(x): 

a CNN that classifies true image X as 1 and fake image G(z) as 0 

c. Training algorithm: 

i. for K steps: train D to maximize (G keeps still), estimating probability by fitting a gaussian parzen window



where x(i) is true image; z(i) is noise vector, D(x) is Prob(x is true image) 

ii. D keeps still, train G to maximize 



iii. repeat step 1, 2 (G can't be trained too much without updating D, otherwise G will collapse too many z to same x) 

2. Changes in this paper (GANs unstable: generates incomprehensible images and noise) 

a. use strided convolutions instead of any spatial pooling ==> allowing nets to learn downsampling 

b. use batch normalization ==> useful to prevent G from collapsing all z into same x 

c. use global average pooling instead of FC  ==> increase stability but hurt convergence speed 

d. use leaky ReLU in D(x) for all layers ==> useful for higher resolution modeling

e. *might be useful: sample deduplication: 3072-128-3072 dropout regularized ReLU autoencoder 

3. Performance: 

Discriminator: 



Generator: