DTC(Deep Temporal Clustering--Fully Unsupervised Learning of Time-Domain Features)论文理解
2020-07-16 15:16
411 查看
DTC网络模型图:
整体网络模型的损失函数为:重构损失+聚类损失,即MSE+KL
一、MSE
经过TAE(temporal autoencoder)层,在输出和输入之间求重构损失MSE,TAE层具体网络类型结构见上图
二、聚类层
1.初始化簇中心:先对所有样本的潜在特征进行complete linkage的层次聚类,再进行k切割(得到k个簇),再对每个簇的元素求平均来获取初始的簇中心。
2.训练过程:①计算xi属于第j个簇的概率,xi的潜在特征zi离第j个簇中心wj越近,则xi属于第j个簇的概率越大;②用loss function更新簇中心,这个loss function使用目标分布p最大化高置信度聚类结果。
3.以两个簇为例:
输入的z计算和每个簇中心w的相似性,再送入Student’s t 分布核来转换为属于每个簇的概率。这里的相似性度量方法有:CID,COR,ACF,EUCL
4.聚类层损失函数:训练聚类层的目的是最小化qij和pij之间的KL散度,所以L=∑i=1n∑j=1kpijlogpijqijL=\sum^n_{i=1}\sum^k_{j=1}p_{ij}log\frac{p_{ij}}{q_{ij}}L=∑i=1n∑j=1kpijlogqijpij,其中n是样本个数,k是类的个数
三、DTC优化过程
簇中心的初始值会较大影响结果,因此这里对TAE进行预训练来获得更有意义的z
步骤:①预训练②预训练后,由complete-linkage的层次聚类初始化簇中心③再用dLedzi\frac{dL_e}{dz_i}dzidLe和dLaedz\frac{dL_{ae}}{dz}dzdLae分别更新AE和簇中心,同时每次采用SGD进行优化时,根据公式(5)更新目标分布p
相关文章推荐
- 《A Review of Unsupervised Feature Learning and Deep Learning for Time-Series Modeling》笔记
- 论文阅读理解 - Deep Learning of Binary Hash Codes for Fast Image Retrieval
- DTCR(Learning Representations for Time Series Clustering)论文理解
- 【Paper Note】Convolutional Clustering for Unsupervised Learning 论文理解
- 论文阅读:Comparative Study of Deep Learning Software Frameworks( caffe、Neon、TensorFlow、Theano、Torch 之比较)
- 单目深度估计论文复现之Unsupervised Learning of Depth and Ego-Motion from Video
- 【论文阅读笔记】Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique
- 多任务学习-An Overview of Multi-Task Learning in Deep Neural Networks论文笔记
- 关于The Limitations of Deep Learning in Adversarial Settings的理解
- The Future of Real-Time SLAM and "Deep Learning vs SLAM" 深度学习与slam
- CVPR 论文阅读与翻译2:图像检索、哈希编码学习、深度哈希:Deep Learning of Binary Hash Codes for Fast Image Retrieval-2015
- Detection of Malicious Code Variants Based on Deep Learning--论文
- Salient Subsequence Learning for Time Series Clustering论文笔记
- Automatic Feature Learning for Glaucoma Detection Based on Deep Learning论文理解
- DeepSense: a Unified Learning Framework for Time-Series Mobile Sensing Data Processing 论文笔记
- 论文阅读:End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for H
- The Future of Real-Time SLAM and “Deep Learning vs SLAM”
- 论文笔记 A Large Contextual Dataset for Classification,Detection and Counting of Cars with Deep Learning
- Unsupervised Learning of Monocular Depth Estimation and Visual Odometry论文阅读
- 论文阅读理解 - Deep Metric Learning via Lifted Structured Feature Embedding