将遗传算法用于神经网络
目前我正在研究一个使用遗传算法来优化神经网络的项目。我确实意识到这可能不是优化它们的最佳方法，但我对两者都很陌生，所以我只是想尝试使用它们。
我的计划如下（可能会发生很多变化）。我的输入神经元将使用一个数据集，该数据集可能具有几乎任何正数（包括最多两位小数，因此实际上它们将是浮点数），但最有可能在 0 到 20000 之间。因为重要性在于如何数字之间比较的是值而不是它们的大小，它们首先除以所有输入值中的最大数。在进入隐藏层之前，他们会将它们乘以权重（任何正或负浮点数）。隐藏层中的每个神经元都会对其所有输入求和，直到计算完成。然后它们将通过物流功能运行并输出。
我的环境是 Visual studio C++ 2010 Express，我使用的是 clr。
我的问题在于遗传算法及其工作原理。这将是调整权重。我的问题是，当它随机改变其中一个权重（突变率）时，它可能会使权重异常高或低，从而在乘以输入并与其他项相加时导致溢出或其他错误。我也不知道如何组织我的染色体。那么，通过选择权重而不是随机位并将其更改为定义范围内的随机数来执行随机化会更好吗？基本上，我正在寻找有关如何组织此操作的建议，而不会导致错误，使值最终太大或太小，同时保持性能。
谢谢，（很抱歉，如果这应该属于理论计算机科学，但我认为它不适合那里）
Currently I am working on a project that would use genetic algorithms to optimize neural networks. I do realize this is probably not the best way to optimize them, but I'm new to both so I just wanted to try using them.
My plans are as follows (subject to change, a lot). My input neurons would be using a data set that could have just about any positive number (including decimals up to two places, so really they would be floating point numbers), but most likely between 0 to 20000. Because the importance is in how the numbers compare to each other in value rather than how big they are, they would first be divided by the highest number of all the values that would be inputted. They would them be multiplied by weights (any positive or negative float) before going to their hidden layer. Each neuron in the hidden layer would be summing all of it's inputs until they were done being calculated. They would then be run through a logistics function and be outputted.
My environment is Visual studio C++ 2010 Express and I'm using the clr.
My problem lies in the genetic algorithm and how it would work. It would be adjusting the weights. My problem is that when it randomly changes a bit in one of the weights (mutation rate), it could make the weights extraordinarily high or low, causing overflow or some other error when multiplied by the input and added with the others. I have no idea how I would organize my chromosomes either. So, would it just be better to perform randomization by selection weights rather than bits at random and changing them to a random number within a defined range? Basically I am looking for suggestions on how to organize this without causing errors with making values end up too big or too small while maintaining performance.
Thanks, (and sorry if this should be in theoretical computer science, but I thought it wouldn't fit in there)
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
众所周知，（人工）神经网络 (ANN) 很难优化，而遗传算法 (GA) 是一种相当好的优化方法（主要是因为其他所有方法的工作效果往往非常有限）。当然，也有一些效果很好的替代方案，但它们的编程和调整更加复杂和微妙（带有模拟退火和学习动量的反向传播）。我知道你做这个项目主要是为了玩这些东西。
您可能想看看进化神经控制器（ENC），这是一个采用遗传（或进化）算法来训练人工神经网络以执行复杂导航任务的领域（例如，行星际空间任务是我的应用之一）亲自研究过）。
对于 ANN 部分，我建议您不要将自己局限于物流功能（我知道 sigmoid 是受到生物神经元的启发，但这并不意味着它们一直是最好的）。还存在许多其他函数，使用后勤函数的部分原因是它们使反向传播更快、更简单。但是，径向基函数也能创造奇迹（IMO，据我所知，人工神经网络最成功的应用都使用径向基函数，即 RBF-NN）。通常，人们使用高斯函数、超球面函数，并且经常使用三角函数（称为模糊网络，这是人工神经网络的另一大类）。
至于 GA，出于您提到的原因，我不会推荐您所描述的突变类型（即翻转位）。人们在处理实值基因时不会使用这种突变。一种非常简单的突变方法就是决定（以一定的概率）对个体进行突变，然后选择其基因中要突变的一个元素，然后简单地使用随机数生成器（rand()）生成一个新的基因元素来替换它。这样，您就可以限制生成的基因元件的规模，以避免导致个体退化的问题（即一个完全错误的基因元件可以使整个个体变得无用）。基因是什么？嗯，对于人工神经网络，通常是一个包含网络中所有神经元的所有权重的大向量。你可以猜测，如果神经元数量太多，人们很少会应用遗传算法。我还建议您使用锦标赛选择来选择要复制的个体。至于交叉（即混合两个父母生成一个孩子），只需保持权重的顺序，并为孩子的每个元素以相同的概率随机从任一父母那里选择一个权重。
我个人已经完成了上面描述的操作，并且它对于某些问题非常有效（减小尺寸和高复杂性，即没有明显的最佳解决方案）。
最后，不要指望它会那么容易工作。通常，它需要的种群规模和世代数量远高于您的预期（毕竟，进化是一个非常缓慢的过程！）。所以，不要尝试用 10 个人组成的群体运行 50 代，然后悲伤地说“哦，我想这行不通……”。当然，可以在人口中的数千个个体和几千到十万代的数量级上进行更多尝试，这取决于您所应用的问题的规模。
(Artificial) Neural Networks (ANNs) are notoriously difficult to optimize, and genetic algorithms (GAs) are a reasonably good approach to doing so (mainly because everything else tends to be very limited in how well it can work). Of course there are alternatives that work well too, but they are more complicated and subtle to program and tune correctly (back-propagation with simulated annealing and learning momentum). And I understand your are doing this project mostly to play around with those things.
You might want to look at Evolutionary Neuro-controllers (ENC), this is a field where genetic (or evolutionary) algorithms are employed to train ANNs for complex navigation tasks (e.g. inter-planetary space missions is one of the applications of that I have personally done research on).
For the ANN part, I would suggest that you don't limit yourself to logistics functions (I know that the sigmoid is inspired from biological neurons, but that doesn't mean they are the best all the time). Many other functions exist as well, logistics functions are used in part because they make back-propagation much faster and simpler to do. But, Radial-Basis functions work wonders as well (IMO and from what I have seen, most successful applications of ANNs used Radial-Basis functions, i.e. RBF-NN). Typically, people use either Gaussian functions, hyper-spherical functions and very often, triangular functions (called Fuzzy Networks, another huge class of ANNs).
As for GAs, I would not recommend that type of mutation you are describing (i.e. to flip bits) for the reasons that you mentioned. People don't use this kind of mutation when dealing with real-valued genes. One very easy mutation method is just to decide (with some probability) to mutate an individual, then pick one element of its gene to be mutated and then simply generate a new gene element to replace it using a random number generator (rand()). With this, you can limit the scale of the generated gene elements to avoid problems of turning your individual degenerate (i.e. one completely wrong gene element can make the entire individual useless). What are the genes? Well, for a ANN, typically a large vector containing all the weights of all the neurons in your net. You can guess that people rarely apply GAs if the number of neurons is too large. I would also recommend that you use Tournament Selection for selecting individuals for reproduction. As for cross-over (i.e. mixing two parents to make a child), just keep the order of the weights and pick for each element of the child a weight from either parent at random with equal probability.
I have personally done what I described above and it works very well for certain problems (reduced size and high complexity, i.e. no obvious optimal solution).
Finally, don't expect it to work that easily. Typically, it will require a population size and a number of generation that is much much higher than what you would expect (after all, evolution is a very slow process!). So, don't try a population of 10 individuals and run for 50 generations, and sadly say "OH I guess it doesn't work...". Try more in the order of thousands of individuals in the population and several thousands to a hundred thousand generations, depending on the scale of the problem your are applying it to, of course.
你的问题在于染色体的表示。这被称为汉明悬崖问题。您可以使用 格雷码 来表示不存在汉明悬崖问题的染色体
Your problem lies in chromosome representation. It's known as Hamming Cliff Problem. You can use Gray Code for chromosome representation which doesn't have Hamming Cliff Problem