[深度学习论文笔记][Image Reconstruction] Understanding Deep Image Representations by Inverting Them

Mahendran, Aravindh, and Andrea Vedaldi. “Understanding deep image representations by inverting them.” 2015 IEEE conference on computer vision and pattern recognition 
(CVPR). IEEE, 2015. (Citations: 142).

1 Motivation

Given an intermediate layer’s representation of an image, to which extent is it possible to reconstruct the image itself?

Note that the representations collapse irrelevant differences in images (e.g. illumination or viewpoint), so that the representation should not be uniquely invertible.

2 Idea

Starting from random noise, find an image whose representations best matches the given one.



Where A is the given representation, Â is the one of the image X. (X) is the regulariser

capturing a natural image prior.

Note after optimization, training set mean image should be added on X.

3 Regularisers

Discriminatively-trained representations may discard a significant amount of low-level image statistics as these are usually not interesting for high-level tasks.

3.1 p-norm



By choosing a relatively large exponent (p = 6 is used in the experiments) the range of the image is encouraged to stay within a target interval instead of diverging.

3.2 Total Variation (TV)

It encourages images to consist of piece-wise constant patches.



Choice of β

• When CNN has max pooling layer, β = 1 will lead to spikes in reconstruction since TV counts overall amount of intensity change.

• β < 1 will eacerbate this issue.

• β > 1 will remove skipes, but the image will be washed out as edges are penalized more. 

4 Balancing the Different Terms



Divided the loss by ||A||^2 F will let it in the region of 0, 1).

5 Result

See Fig. All the convolutional layers maintain a photographically faithful representation of the image, although with increasing fuzziness. The fc layers are inverted back to a

composition of parts similar but not identical to the ones found in the original image. FC layers loses information about the location of the object, but retain information about some discriminative features of the objects.