是否可以强制 cPickle 使用广度优先而不是深度优先递归?
我意识到答案很可能是否定的!
基本上,我有一个图(节点和边类型),它代表一个正方形网格;每个节点对象都包含对该节点具有边缘的每个其他节点的引用,这似乎意味着当使用 cPickle.dump 序列化图时,它会以深度优先的方式遍历图中的每个节点,这意味着对于一个井- 表示 16x16 网格的连通图,它有效地将其视为 256 层深的数据结构。这意味着较大的网格很快就会超出默认的最大 Python 递归深度,特别是因为实验表明,似乎需要对堆栈进行大约 4 次调用才能进入数据结构的额外级别。
问题是,我还有一个字典中的字典,它以允许我使用笛卡尔坐标来查找特定节点的方式引用该图(例如“node=nodes[3][6]”)。所以从概念上讲,它根本不是一个高度嵌套的数据结构,它是一个相当扁平的数据结构,恰好有很多横向引用,但似乎 cPickle 完全是深度优先的(据我所知,这是迄今为止最简单的方法)工作)。
现在,我了解 sys.setrecursionlimit(),并且我已经做了一些实验来找出我需要为图形大小设置限制有多大,所以这是“最简单”的选项。我知道我可以退出节点到节点的链接并依靠字典的字典来维护网格和单独的平面结构来维护边缘权重,但有多种原因我想避免- 尤其是节点到节点的链接允许更直观地使用数据结构。我相信根据我所读到的内容,我应该能够提供自己的 __getstate__ 和 __setstate__ 实现并覆盖 pickling 功能,但显然这是一个不小的数量的工作。如果有一种方法可以让 cPickle(或 pickle,我不挑剔!)使用广度优先遍历,它应该可以非常简单地解决问题!
I realise the answer is very likely to be 'no'!
Basically, I have a graph (the nodes and edges kind) which represents a grid of squares; each node object contains references to every other node that this node has an edge to, which seems to mean that when the graph is serialised using cPickle.dump it traverses every node in the graph in a depth-first fashion, meaning that for a well-connected graph representing a 16x16 grid, it's effectively treating it as a 256-level-deep data structure. This means that larger grids very quickly overrun the default maximum Python recursion depth, particularly since experimentation suggests it seems to take about 4 calls on the stack to go an extra level into the data structure.
The thing is, I also have a dict-of-dicts that refers into this graph in such a way as to allow me to use cartesian coordinates to find particular nodes (e.g. "node = nodes[3][6]"). So conceptually, it's not a highly-nested data structure at all, it's a fairly flat one which happens to have a lot of sideways references, but it seems that cPickle works entirely depth-first (which I understand is by far the easiest way to work).
Now, I know about sys.setrecursionlimit(), and I've done some experimentation to find out how big I'd need to set the limit for what size of graph, so that's the 'easiest' option. I'm aware that I could just quit the node-to-node links and rely on the dict-of-dicts to maintain the grid and a separate flat structure to maintain edge weightings, but there are various reasons I'd like to avoid that - not least that the node-to-node links allow for a more intuitive use of the data structure. I believe from what I've read that I should be able to provide my own implementations of __getstate__
and __setstate__
and override the pickling functionality, but obviously that's a non-trivial amount of work. If there were a way to get cPickle (or pickle, I'm not fussy!) to use a breadth-first traversal, it should solve the problem pretty simply!
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
编写一个合适的 __getstate__() 方法似乎并没有那么复杂。尝试类似的方法,
这将腌制
Node
实例的所有属性,但neighbours
属性除外,我认为该属性包含指向邻居的链接。 (您不需要__setstate__()
方法。)在取消整个图之后,您将必须重新创建到所有节点上的邻居的链接,但这也不应该那么困难。
Writing a suitable
__getstate__()
method does not seem to be that complicated after all. Try something likeThis will pickle all attributes of
Node
instance except for theneighbours
attribute, which I assume to contain the links to the neighbours. (You don't need a__setstate__()
method.)After unpickling the whole graph, you will have to recreate the links to the neighbours on all nodes, but this shouldn't be that difficult either.