神经网络真的是废弃软件吗？
我计划使用神经网络来近似强化学习算法中的值函数。我想这样做是为了在我如何表示状态和行动方面引入一些概括性和灵活性。
现在，在我看来，神经网络是实现这一目标的正确工具，但由于我不是人工智能专家，因此我对此的了解有限。特别是，如今神经网络似乎正在被其他技术所取代，例如支持向量机，但我不确定这是否是一个时尚问题，或者神经网络是否存在一些真正的限制，可能会毁掉我的方法。您有什么建议吗？
谢谢，
通努兹
I am planning to use neural networks for approximating a value function in a reinforcement learning algorithm. I want to do that to introduce some generalization and flexibility on how I represent states and actions.
Now, it looks to me that neural networks are the right tool to do that, however I have limited visibility here since I am not an AI expert. In particular, it seems that neural networks are being replaced by other technologies these days, e.g. support vector machines, but I am unsure if this is a fashion matter or if there is some real limitation in neural networks that could doom my approach. Do you have any suggestion?
Thanks,
Tunnuz
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
确实，神经网络不再像以前那样流行，但它们还没有消亡。它们失宠的一般原因是支持向量机的兴起，因为它们在全球范围内收敛并且需要较少的参数规格。
然而，SVM 的实施起来非常繁琐，并且不能像 ANN 那样自然地推广到强化学习（SVM 主要用于离线决策问题）。
如果您的任务适合，我建议您坚持使用人工神经网络，因为在强化学习领域，人工神经网络在性能方面仍然处于领先地位。
这是一个很好的起点；只需查看标题为“时间差异学习”的部分，因为这是人工神经网络解决强化学习问题的标准方法。
但需要注意的是：机器学习的最新趋势是通过 bagging 或 boosting。虽然我在强化学习中没有看到太多这种情况，但我确信采用这种策略仍然比单独的人工神经网络更强大。但除非您确实需要世界一流的性能（这就是赢得 Netflix 竞赛的原因），否则我会避开这种极其复杂的技术。
It's true that neural networks are no longer in vogue, as they once were, but they're hardly dead. The general reason for them falling from favor was the rise of the Support Vector Machine, because they converge globally and require fewer parameter specifications.
However, SVMs are very burdensome to implement and don't naturally generalize to reinforcement learning like ANNs do (SVMs are primarily used for offline decision problems).
I'd suggest you stick to ANNs if your task seems suitable to one, as within the realm of reinforcement learning, ANNs are still at the forefront in performance.
Here's a great place to start; just check out the section titled "Temporal Difference Learning" as that's the standard way ANNs solve reinforcement learning problems.
One caveat though: the recent trend in machine learning is to use many diverse learning agents together via bagging or boosting. While I haven't seen this as much in reinforcement learning, I'm sure employing this strategy would still be much more powerful than an ANN alone. But unless you really need world class performance (this is what won the netflix competition), I'd steer clear of this extremely complex technique.
在我看来，神经网络正在卷土重来。例如，今年ICML 2011 上有一堆关于神经网络的论文。我绝对不会认为它们是废弃软件。话虽如此，我不会将它们用于强化学习。
It seems to me that neural networks are kind of making a comeback. For example, this year there were a bunch of papers at ICML 2011 on neural networks. I would definitely not consider them abandonware. That being said, I would not use them for reinforcement learning.
神经网络是近似复杂函数的一种不错的通用方法，但它们很少是任何特定学习任务的最佳选择。它们很难设计，收敛速度慢，并且会陷入局部极小值。
如果您没有神经网络方面的经验，那么您可能会更乐意使用更直接的泛化 RL 方法，例如粗略编码。
Neural networks are a decent general way of approximating complex functions, but they are rarely the best choice for any specific learning task. They are difficult to design, slow to converge, and get stuck in local minima.
If you have no experience with neural networks, then you might be happier to you use a more straightforward method of generalizing RL, such as coarse coding.
从理论上讲，已经证明神经网络可以近似任何函数（给定无限数量的隐藏神经元和必要的输入），所以我不认为神经网络将永远被抛弃。
SVM 很棒，但它们不能用于所有应用，而神经网络可以用于任何目的。
将神经网络与强化学习结合使用是标准且众所周知的，但要小心绘制和调试神经网络的收敛，以检查其是否正常工作，因为众所周知，神经网络很难正确实现和学习。
还要非常小心地对待您向神经网络提供的问题的表示（即：输入节点）：您或专家能否根据您向网络提供的输入来解决问题？很多时候，实现神经网络的人不会为神经网络提供足够的信息来进行推理，这种情况并不少见，所以要小心。
Theoretically it has been proved that Neural Networks can approximate any function (given an infinite number of hidden neurons and the necessary inputs), so no I don't think the neural networks will ever be abandonwares.
SVM are great, but they cannot be used for all applications while Neural Networks can be used for any purpose.
Using neural networks in combination with reinforcement learning is standard and well-known, but be careful to plot and debug your neural network's convergence to check that it works correctly as neural networks are notoriously known to be hard to implement and learn correctly.
Be also very careful about the representation of the problem you give to your neural network (ie: the inputs nodes): could you, or could an expert, solve the problem given what you give as inputs to your net? Very often, people implementing neural networks don't give enough informations for the neural net to reason, this is not so uncommon, so be careful with that.