机器学习中使用的神经网络(三)
2015-05-02 21:58
274 查看
神经元的简单模型
Idealized neurons
• To model things we have to idealize them (e.g. atoms)
– Idealization removes complicated details that are not essential for understanding the main principles.
– It allows us to apply mathematics and to make analogies to other, familiar systems.
– Once we understand the basic principles, its easy to add complexity to make the model more faithful.
• It is often worth understanding models that are known to be wrong (but we must not forget that they are wrong!)
– E.g. neurons that communicate real values rather than discrete spikes of activity.
Linear neurons
• These are simple but computationally limited
– If we can make them learn we may get insight into more complicated neurons
Binary threshold neurons 二值阈值神经元
McCulloch-Pitts (1943): influenced Von Neumann.
– First compute a weighted sum of the inputs.
– Then send out a fixed size spike of activity if the weighted sum exceeds a threshold.
– McCulloch and Pitts thought that each spike is like the truth value of a proposition and compute the truth value of another proposition!
There are two equivalent ways to write the equations for a binary threshold neuron:
Rectified Linear Neurons
(sometimes called linear threshold neurons)
They compute a linear weighted sum of their inputs.
The output is a non-linear function of the total input
Sigmoid neurons 这个神经元经常使用
These give a real-valued output that is a smooth and bounded function of their total input.
– Typically they use the logistic function
– They have nice smooth derivatives, the derivatives change continuously and they're nicely behaved and they make it easy to do learning.
Stochastic binary neurons 随机二进制神经元
These use the same equations as logistic units.
– But they treat the output of the logistic as the probability of producing a spike in a short time window.
Instead of outputtiing that probability as a real number they actually make a probabilistic decision, and so what they acutally output is either a one or a zero. They're intrisically random. So they're treating the P as the probability of producing a one, not as a real number.
• We can do a similar trick for rectified(改正的) linear units:
– The output is treated as the Poisson rate for spikes.
So the rectified linear unit determines the rate, but intrinsic randomness in the unit determines when the spikes are actually produced.
Idealized neurons
• To model things we have to idealize them (e.g. atoms)
– Idealization removes complicated details that are not essential for understanding the main principles.
– It allows us to apply mathematics and to make analogies to other, familiar systems.
– Once we understand the basic principles, its easy to add complexity to make the model more faithful.
• It is often worth understanding models that are known to be wrong (but we must not forget that they are wrong!)
– E.g. neurons that communicate real values rather than discrete spikes of activity.
Linear neurons
• These are simple but computationally limited
– If we can make them learn we may get insight into more complicated neurons
Binary threshold neurons 二值阈值神经元
McCulloch-Pitts (1943): influenced Von Neumann.
– First compute a weighted sum of the inputs.
– Then send out a fixed size spike of activity if the weighted sum exceeds a threshold.
– McCulloch and Pitts thought that each spike is like the truth value of a proposition and compute the truth value of another proposition!
There are two equivalent ways to write the equations for a binary threshold neuron:
Rectified Linear Neurons
(sometimes called linear threshold neurons)
They compute a linear weighted sum of their inputs.
The output is a non-linear function of the total input
Sigmoid neurons 这个神经元经常使用
These give a real-valued output that is a smooth and bounded function of their total input.
– Typically they use the logistic function
– They have nice smooth derivatives, the derivatives change continuously and they're nicely behaved and they make it easy to do learning.
Stochastic binary neurons 随机二进制神经元
These use the same equations as logistic units.
– But they treat the output of the logistic as the probability of producing a spike in a short time window.
Instead of outputtiing that probability as a real number they actually make a probabilistic decision, and so what they acutally output is either a one or a zero. They're intrisically random. So they're treating the P as the probability of producing a one, not as a real number.
• We can do a similar trick for rectified(改正的) linear units:
– The output is treated as the Poisson rate for spikes.
So the rectified linear unit determines the rate, but intrinsic randomness in the unit determines when the spikes are actually produced.
相关文章推荐
- 机器学习中使用的神经网络第一讲笔记:Introduction
- 机器学习中使用的神经网络(七)
- stanford coursera 机器学习编程作业 exercise 3(使用神经网络 识别手写的阿拉伯数字(0-9))
- 机器学习系列直播--使用对抗神经网络(GANs)生成猫
- 机器学习中使用的神经网络第二讲笔记
- 机器学习中使用的神经网络第二讲笔记:神经网络的结构和感知机
- 机器学习中使用神经网络第三讲笔记
- stanford coursera 机器学习编程作业 exercise4--使用BP算法训练神经网络以识别阿拉伯数字(0-9)
- 机器学习中使用的神经网络第九讲笔记
- 机器学习中使用的神经网络(五)
- 使用机器学习预测天气(第三部分神经网络)
- 机器学习中使用的神经网络(四)
- 机器学习中使用的神经网络第五讲笔记
- (转)机器学习系列直播--使用对抗神经网络(GANs)生成猫
- 机器学习的大局观:使用神经网络和TensorFlow来对文本分类
- 机器学习中使用的神经网络(一)
- 机器学习中使用的神经网络第四讲笔记
- 机器学习中使用的神经网络第六讲笔记
- 机器学习中使用的神经网络(二)
- 神经网络基础 - PyBrain机器学习包的使用