[深度学习论文笔记][Visualizing] Understanding Neural Networks Through Deep Visualization

Yosinski, Jason, et al. “Understanding neural networks through deep visualization.” arXiv preprint arXiv:1506.06579 (2015). (Citations: 65).

1 Optimization For Any Arbitary Neuron



2 Three Forms of Regularizations
This paper proposed some more explicit forms of regularization. Different hyperparameter combinations of these regularizations produce different styles of reconizable images.

.2.1 Clipping Pixels with Small Contribution

Contribution can be measured as



Where Ã(d, i, j) is the activation with a certain pixel of X be set to zero.

However, this approach is slow since it needs to forward pass for every pixel. So we approximate Ã(d, i, j) by



So



Finally, we sum across channels to find pixels with small distribution in either direction, positive or negative, and set those pixels to zero by thresholding.

2.2 Blur X a Bit

Producing images via gradient ascent tends to produce examples with high frequency information. While these images cause high activations, they are neither realistic nor  interpretable. As a result, we blur the image with Gaussian kernel every several steps
due to the computational effectiveness.

2.3 Take Any Pixel With Small Norm to Zero

CNN tends to give X non-zero everywhere to contribute in whatever small way they can to ultimately raise the chosen unit’s activation. We instead show only the main object, letting other regions be exactly zero by thresholding if they are not needed.

3 References

[1]. Deep Visualization Toolbox. https://www.youtube.com/watch?v=AgkfIQ4IGaM.