generative VS discrimi…

总是听到这两个术语，但是又一直不清楚它们最本质的区别。今天花了一小点时间来彻底的弄清楚了。得到的结论如下：

Discriminative Model是判别模型，又可以称为条件模型，或条件概率模型。

Generative Model是生成模型，又叫产生式模型。

二者的本质区别是

discriminative model 估计的是条件概率分布(conditional
distribution)p(class|context)

generative model 估计的是联合概率分布（joint probability
distribution）p()

常见的Generative Model主要有：

– Gaussians, Naive Bayes, Mixtures of multinomials

– Mixtures of Gaussians, Mixtures of experts, HMMs

– Sigmoidal belief networks, Bayesian networks

– Markov random fields

常见的Discriminative Model主要有：

– logistic regression

– SVMs

– traditional neural networks

– Nearest neighbor

Successes of Generative Methods：

? NLP

– Traditional rule-based or Boolean logic systems

Dialog and Lexis-Nexis) are giving way to statistical

approaches (Markov models and stochastic context

grammars)

? Medical Diagnosis

– QMR knowledge base, initially a heuristic expert

systems for reasoning about diseases and symptoms

been augmented with decision theoretic formulation

? Genomics and Bioinformatics

– Sequences represented as generative HMMs

主要应用Discriminative Model：

? Image and document classification

? Biosequence analysis

? Time series prediction

Discriminative Model缺点：

? Lack elegance of generative

– Priors, structure, uncertainty

? Alternative notions of penalty functions,

regularization, kernel functions

? Feel like black-boxes

– Relationships between variables are not explicit

and visualizable

Bridging Generative and Discriminative：

? Can performance of SVMs be combined

elegantly with flexible Bayesian statistics?

? Maximum Entropy Discrimination marries

both methods

– Solve over a distribution of parameters (a

distribution over solutions)

转自http://billlangjun.blogspot.com/2006/09/discriminative-vs-generative-model.html