Unsupervised Image-to-Image Translation Networks --- Reading Writing
2017-03-03 19:36
681 查看
Unsupervised Image-to-Image Translation Networks --- Reading Writing
2017.03.03
Motivations: most existing image to image translation algorithms are all need image pairs as training data for deep neural network, such as CGANs or VAEs. But in some cases, it is rather difficult to collect such training data. For example, the night and day image pairs, the perfect aligned thermal RGB image pairs, or sunning rainning, fogging, et al, which provide us a new challenging problem:
How to do image to image translation in a unsupervised fashion which do not need aligned image pairs ?
This paper proposed the UNIT framework (UNsupervised Image-to-image Translation network) to deal with this problem which combine VAE and GANs. The whole framework can be described as the following figures which seems complex but rather easy to understand.
There are two most important assumptions about the proposed framework:
1. we assume that the relationship between X1 and X2 does not only exist at the image level but also at the level of local patches or regions.
2. for any given images x1 and x2, there exists a common underlying representation z, such that we can cover both images from this underlying representation from each of the two input images.
VAEs: the encoder-generator pair {E1, G1} constitutes a VAE for the X1 domain, termed VAE1. Another pair of {E2, G2} constitutes a VAE for the X2 domain VAE2.
Weight-sharing : we enforce a weight-sharing constraint to relate the representations in the two VAEs.
GANs : two GANs are used to output the two domains.
Experiments:
2017.03.03
Motivations: most existing image to image translation algorithms are all need image pairs as training data for deep neural network, such as CGANs or VAEs. But in some cases, it is rather difficult to collect such training data. For example, the night and day image pairs, the perfect aligned thermal RGB image pairs, or sunning rainning, fogging, et al, which provide us a new challenging problem:
How to do image to image translation in a unsupervised fashion which do not need aligned image pairs ?
This paper proposed the UNIT framework (UNsupervised Image-to-image Translation network) to deal with this problem which combine VAE and GANs. The whole framework can be described as the following figures which seems complex but rather easy to understand.
There are two most important assumptions about the proposed framework:
1. we assume that the relationship between X1 and X2 does not only exist at the image level but also at the level of local patches or regions.
2. for any given images x1 and x2, there exists a common underlying representation z, such that we can cover both images from this underlying representation from each of the two input images.
VAEs: the encoder-generator pair {E1, G1} constitutes a VAE for the X1 domain, termed VAE1. Another pair of {E2, G2} constitutes a VAE for the X2 domain VAE2.
Weight-sharing : we enforce a weight-sharing constraint to relate the representations in the two VAEs.
GANs : two GANs are used to output the two domains.
Experiments:
相关文章推荐
- Image-to-Image Translation with Conditional Adversarial Networks
- 『论文阅读』Image-to-Image Translation with Conditional Adversarial Networks
- 每日论文image-to-Image Translation with Conditional Adversarial Networks
- Image-to-Image Translation with Conditional Adversarial Networks 论文翻译
- Image-to-Image Translation with Conditional Adversarial Networks
- Image-to-Image Translation with Conditional Adversarial Networks论文学习
- 论文Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
- Image-to-Image Translation with Conditional Adversarial Networks_201611
- ICCV2017论文“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks” 阅读笔记
- Unsupervised Image-to-Image Translation Networks---VAE+GAN+Cycle
- Learning to Compare Image Patches via Convolutional Neural Networks
- 基于2-channel network的图片相似度判别_2015Learning to Compare Image Patches via Convolutional Neural Networks
- StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks 论文笔记
- 论文学习:Learning to Compare Image Patches via Convolutional Neural Networks
- GAN在Image To Image translation 和Inverse Problem中的应用
- Howto Setup yum repositories to update or install package from ISO CDROM Image
- 接入京东开普勒报错: '+[UIImage imagePathCustomJDKeplerBundle:type:]: unrecognized selector sent to class
- Learning How To Code Neural Networks
- 找到网页的某一区域,截图,并保存 HTML to Image
- 论文读书笔记-Using neural network to combine measures of word semantic similarity for image annotation