Python itertools.combinations() 内存问题
我正在处理大量的物品组合（来自《英雄联盟》），大约 7200 万个，所有这些组合都被输入一个函数来计算它们的效益。
我们正在努力寻找最佳的组合。
忽略从算法上来说可能有更好的方法来做到这一点的事实，有人能告诉我为什么会出现内存错误吗？
allpossiblei = itertools.combinations(items.keys(),5)
maxc = 0
i = 0
for combo in allpossiblei:
icombo = [items[name] for name in combo]
res, tcost = calcStats(icombo, 0.658,100,100)
if res > maxc :
maxc = res
print str(res) + " " + str(res/tcost)
print combo
print float(i)/79208745.0
if i % 500000 == 0:
print str(float(i)/79208745.0) + "\n \n"
gc.collect()
i = i + 1
除了使用局部变量进行算术运算之外，calcStats 不执行任何操作。
这会迅速耗尽 2GB 以上的内存，并在大约 5 分钟内退出。我认为 itertools 应该提供一个不会消耗大量内存的生成器？我什至加入了 gc.collect() 语句，但它似乎不起作用。有什么想法吗？
I'm processing a huge number of combinations of items (from League of Legends), about 72 million, all of which are fed into a function that calculates how beneficial they are.
We're trying to find the best possible combination.
Ignoring the fact that there might be better ways, algorithmically speaking, to do this, can anyone tell me why I'm getting a memory error?
allpossiblei = itertools.combinations(items.keys(),5)
maxc = 0
i = 0
for combo in allpossiblei:
icombo = [items[name] for name in combo]
res, tcost = calcStats(icombo, 0.658,100,100)
if res > maxc :
maxc = res
print str(res) + " " + str(res/tcost)
print combo
print float(i)/79208745.0
if i % 500000 == 0:
print str(float(i)/79208745.0) + "\n \n"
gc.collect()
i = i + 1
calcStats doesn't do anything except arithmetic using local variables.
This rapidly eats up 2gb+ of memory and quits in about 5 minutes. I thought itertools was supposed to provide a generator that wouldn't use up a ton of memory? I even threw in that gc.collect() statement but it doesn't seem to work. Any ideas?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
组合通过消耗提供的整个迭代器来创建池。没有办法解决这个问题。请参阅该函数的文档中的伪代码： http://docs.python.org /library/itertools.html#itertools.combinations
Combinations creates a pool by consuming the whole iterator provided. There is no way around that. See the pseudocode in the docs for the function: http://docs.python.org/library/itertools.html#itertools.combinations