What is the difference between a Generative and Discriminative Algorithm?
2012-09-04 18:57
507 查看
Let's say you have input data x and you want to classify the data into labels y. A generative model learns the joint probability distribution
a discriminative model learns the conditional probability distribution
which you should read as 'the probability of y given x'.
Here's a really simple example. Suppose you have the following data in the form (x,y):
If you take a few minutes to stare at those two matrices, you will understand the difference between the two probability distributions.
The distribution
the natural distribution for classifying a given example
a class
which is why algorithms that model this directly are called discriminative algorithms. Generative algorithms model
which can be tranformed into
applying Bayes rule and then used for classification. However, the distribution
also be used for other purposes. For example you could use
From the description above you might be thinking that generative models are more generally useful and therefore better, but it's not as simple as that. This
paper is a very popular reference on the subject of discriminative vs. generative classifiers, but it's pretty heavy going. The overall gist is that discriminative models generally outperform generative models in classification tasks.
原文:http://stackoverflow.com/questions/879432/what-is-the-difference-between-a-generative-and-discriminative-algorithm
p(x,y)and
a discriminative model learns the conditional probability distribution
p(y|x)-
which you should read as 'the probability of y given x'.
Here's a really simple example. Suppose you have the following data in the form (x,y):
(1,0), (1,0), (2,0), (2, 1)
p(x,y)is
y=0 y=1 ----------- x=1 | 1/2 0 x=2 | 1/4 1/4
p(y|x)is
y=0 y=1 ----------- x=1 | 1 0 x=2 | 1/2 1/2
If you take a few minutes to stare at those two matrices, you will understand the difference between the two probability distributions.
The distribution
p(y|x)is
the natural distribution for classifying a given example
xinto
a class
y,
which is why algorithms that model this directly are called discriminative algorithms. Generative algorithms model
p(x,y),
which can be tranformed into
p(y|x)by
applying Bayes rule and then used for classification. However, the distribution
p(x,y)can
also be used for other purposes. For example you could use
p(x,y)to generate likely
(x,y)pairs.
From the description above you might be thinking that generative models are more generally useful and therefore better, but it's not as simple as that. This
paper is a very popular reference on the subject of discriminative vs. generative classifiers, but it's pretty heavy going. The overall gist is that discriminative models generally outperform generative models in classification tasks.
原文:http://stackoverflow.com/questions/879432/what-is-the-difference-between-a-generative-and-discriminative-algorithm
相关文章推荐
- Multi-thread: What is the difference between OpenMP and MPI
- CareerCup What is the difference between a computers heap and it's stack?
- What is the difference between a pack slip and a pick slip?
- What is the difference between CDATA and PCDATA?
- What is the difference between XML and C or C ?
- What is the difference between XHTML MP, XHTML Basic, WML, i-mode, and HDML?
- What exactly is the difference between WndProc and DefaultWndProc?
- What exactly is the difference between “pass by reference” in C and in C++?
- What is the difference between ./ and sh to run a script?
- overload与override的区别 What is the difference between overload and override
- What is the difference between error active and error passive?
- what is the difference between zImage (kernel) and boot.img
- What is the difference between --save and --save-dev?
- What is the difference between const and static readonly?(const 和 static readonly 的区别?)
- Multi-thread: What is the difference between OpenMP and MPI
- What is the difference between a pack slip and a pick slip?
- In Linux, what is the difference between “buffers” and “cache” reported by the free command?
- What is the difference between /dev/ttyUSB and /dev/ttyACM?
- What is the difference between char s[] and char *s?
- what is the difference between processes and threads