神经网络定义
我正在编写神经网络课程，遇到了两个我不理解的概念。谁能告诉我偏差和动量是什么以及它的作用
I am writing a neural network class and have come across two concepts I don't understand. Would anyone be able to tell me what the bias and momentum is and does
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
偏差是给予神经元的恒定输入。例如，在正常的前馈网络中，您可能有 2 个输入单元、2 个隐藏单元和 1 个输出单元。除了来自输入单元的输入之外，恒定的偏差值（假设为 1）也将进入隐藏单元和输出单元。
动量是在学习开始时使用的额外学习率，以使学习速度更快。例如，学习误差最初通常非常大，因此您以较高的动力开始并更积极地调整权重。随后，在学习过程中，随着错误的减少，动量也会减少，因此您学习的速度会更慢，但超出目标的可能性会较小。
Bias is a constant input given to neurons. e.g. in a normal feed forward network, you might have 2 input units, 2 hidden units and 1 output unit. a constant bias value (let's say 1) will go into the hidden and output units in addition to the input from the input units.
Momentum is the additional learning rate used at the beginning of learning to make learning faster. e.g. learning error is usually initially very large, so you start with high momentum and adjust weights more aggressively. later on during learning as your error decreases, momentum should also decrease so you learn more slowly but you'll be less likely to overshoot the target.
偏差允许神经元接受更广泛的输入值。动量可以被认为是梯度下降过程中的步长。
在典型的节点中，偏差和前一层的所有输入都会被加权、求和，然后压缩为输出值。压缩函数以零为中心，当加权和变得非常正或非常负时，其灵敏度会急剧下降。然而，有时您希望挤压的敏感部分位于输入的某个区域，而不是在零附近。偏差输入允许学习算法改变节点的响应来实现这一点。
除了 Charles Ma 所描述的之外，动量还可以帮助学习算法跨越局部最小值以找到更好的解决方案。
The bias allows the neuron to accept a wider range of input values. Momentum can be thought of as step size during the gradient decent.
In a typical node the bias and all the inputs from previous layer are weighted, summed, and then squashed to the output value. The squashing function is centered around zero and dramatically diminishes in sensitivity as the weighted sum becomes very positive or very negative. However, sometimes you want the sensitive part of the squashing to be at some region of the input other then right around zero. The bias input allows the learning algorithm to shift a node's response to accomplish that.
In addition to what Charles Ma described, momentum can also help carry the learning algorithm across a local minimum to find a better solution.
考虑偏差的直观方法就像线性方程中的 b 项。定义为：
y=mx + b
其中 m 项是斜率，b 告诉您直线与 y 轴的交点。如果增加或减少此项，该线将沿 y 轴上下移动。
神经网络创建一个超平面来分离集群，因此偏差会在训练过程中上下移动它，直到它收敛到最小值，并希望找到适合您的训练样本的适当匹配。
如前所述，该时刻将减少梯度下降期间的振荡。
它将在训练开始时增加步长，并在训练结束时减小步长。您可以将其视为在负梯度方向上朝着最小值迈出更大的步，并在接近最小值时迈出更小的步。这有几个好处：
An intuitiv way to think of the bias is like the b term in a linear equation. Which is defined as:
y=mx + b
The m term is the slope and b tells you where the line intercepts with the y-axis. If you increase or decrease this term, the line will move up and down the y-axis.
A neural network creates a hyperplane to separate clusters, hence the bias will move it up and down during training until it converges to a minima and hopefully found a proper match for your training samples.
The moment, as mantioned before, will reduce the oscillation during gradient descent.
It will increase the step size in the beginning of training and reduce it towards the end. You can think of it as taking bigger steps in the negativ gradient direction, towards a minima, and taking smaller steps as getting closer to it. This has a couple of benefits: