展平不规则(任意嵌套)的列表列表
是的,我知道这个主题之前已经讨论过:
- Python idiom to chain (flatten) anfinite iterable of Finite iterables?
- 在 Python 中展平浅列表
- 展平序列序列的理解?
- 如何从列表列表中制作平面列表?
但据我所知,所有解决方案,除了一个,在像 [[[1, 2, 3], [4, 5]], 6]
这样的列表上失败,其中所需的输出是 [1, 2, 3, 4, 5, 6]
(或者更好的是迭代器)。
我看到的唯一适用于任意嵌套的解决方案是在这个问题中找到的:
def flatten(x):
result = []
for el in x:
if hasattr(el, "__iter__") and not isinstance(el, basestring):
result.extend(flatten(el))
else:
result.append(el)
return result
这是最好的方法吗?我是不是忽略了什么?有什么问题吗?
Yes, I know this subject has been covered before:
- Python idiom to chain (flatten) an infinite iterable of finite iterables?
- Flattening a shallow list in Python
- Comprehension for flattening a sequence of sequences?
- How do I make a flat list out of a list of lists?
but as far as I know, all solutions, except for one, fail on a list like [[[1, 2, 3], [4, 5]], 6]
, where the desired output is [1, 2, 3, 4, 5, 6]
(or perhaps even better, an iterator).
The only solution I saw that works for an arbitrary nesting is found in this question:
def flatten(x):
result = []
for el in x:
if hasattr(el, "__iter__") and not isinstance(el, basestring):
result.extend(flatten(el))
else:
result.append(el)
return result
Is this the best approach? Did I overlook something? Any problems?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(30)
使用生成器函数可以使您的示例更易于阅读并提高性能。
Python 2
使用
Iterable
ABC 2.6 中添加的 :Python 3
在 Python 3 中,
basestring
不再存在,但元组(str, bytes)
具有相同的效果。此外,yield from
运算符一次从生成器返回一个项目。Using generator functions can make your example easier to read and improve performance.
Python 2
Using the
Iterable
ABC added in 2.6:Python 3
In Python 3,
basestring
is no more, but the tuple(str, bytes)
gives the same effect. Also, theyield from
operator returns an item from a generator one at a time.我的解决方案:
更简洁一点,但几乎相同。
My solution:
A little more concise, but pretty much the same.
使用递归和鸭子类型的生成器(针对 Python 3 进行了更新):
Generator using recursion and duck typing (updated for Python 3):
这是我的递归展平的函数版本,它可以处理元组和列表,并允许您添加任意位置参数的组合。返回一个生成器,它按 arg by arg 的顺序生成整个序列:
用法:
Here is my functional version of recursive flatten which handles both tuples and lists, and lets you throw in any mix of positional arguments. Returns a generator which produces the entire sequence in order, arg by arg:
Usage:
@unutbu 的非递归解决方案的生成器版本,按照 @Andrew 在评论中的要求:
该生成器的稍微简化的版本:
Generator version of @unutbu's non-recursive solution, as requested by @Andrew in a comment:
Slightly simplified version of this generator:
这个版本的
flatten
避免了Python的递归限制(因此可以处理任意深度的嵌套迭代)。它是一个可以处理字符串和任意迭代(甚至无限迭代)的生成器。下面是一些演示其用法的示例:
虽然
flatten
可以处理无限生成器,但它无法处理无限嵌套:This version of
flatten
avoids python's recursion limit (and thus works with arbitrarily deep, nested iterables). It is a generator which can handle strings and arbitrary iterables (even infinite ones).Here are some examples demonstrating its use:
Although
flatten
can handle infinite generators, it can not handle infinite nesting:Pandas 有一个函数可以做到这一点。正如您提到的,它返回一个迭代器。
Pandas has a function that does this. It returns an iterator as you mentioned.
这是另一个更有趣的答案......
基本上,它将嵌套列表转换为字符串,使用正则表达式去除嵌套语法,然后将结果转换回(展平的)列表。
Here's another answer that is even more interesting...
Basically, it converts the nested list to a string, uses a regex to strip out the nested syntax, and then converts the result back to a (flattened) list.
您可以使用
deepflatten
第三方包iteration_utilities
:它是一个迭代器所以你需要迭代它(例如用
list
包装它或在循环中使用它)。它在内部使用迭代方法而不是递归方法,并且它被编写为 C 扩展,因此它比纯 Python 方法更快:我是
iteration_utilities
库的作者。You could use
deepflatten
from the 3rd party packageiteration_utilities
:It's an iterator so you need to iterate it (for example by wrapping it with
list
or using it in a loop). Internally it uses an iterative approach instead of an recursive approach and it's written as C extension so it can be faster than pure python approaches:I'm the author of the
iteration_utilities
library.尝试在 Python 中创建一个可以展平不规则列表的函数很有趣,但这当然就是 Python 的用途(让编程变得有趣)。以下生成器工作得相当好,但有一些注意事项:
它将展平您可能希望单独保留的数据类型(例如
bytearray
、bytes
和str
对象)。此外,该代码还依赖于这样一个事实:从不可迭代对象请求迭代器会引发TypeError
。编辑:
我不同意之前的实现。问题是你不应该能够展平不可迭代的东西。它令人困惑,并且给论证留下了错误的印象。
下面的生成器几乎与第一个生成器相同,但不存在尝试展平不可迭代对象的问题。当给予它一个不适当的参数时,它就会像人们所期望的那样失败。
使用提供的列表测试生成器可以正常工作。然而,当给新代码一个不可迭代的对象时,它会引发一个
TypeError
。下面显示了新行为的示例。It was fun trying to create a function that could flatten irregular list in Python, but of course that is what Python is for (to make programming fun). The following generator works fairly well with some caveats:
It will flatten datatypes that you might want left alone (like
bytearray
,bytes
, andstr
objects). Also, the code relies on the fact that requesting an iterator from a non-iterable raises aTypeError
.Edit:
I disagree with the previous implementation. The problem is that you should not be able to flatten something that is not an iterable. It is confusing and gives the wrong impression of the argument.
The following generator is almost the same as the first but does not have the problem of trying to flatten a non-iterable object. It fails as one would expect when an inappropriate argument is given to it.
Testing the generator works fine with the list that was provided. However, the new code will raise a
TypeError
when a non-iterable object is given to it. Example are shown below of the new behavior.这是一个简单的函数,可以展平任意深度的列表。没有递归,避免堆栈溢出。
Here's a simple function that flattens lists of arbitrary depth. No recursion, to avoid stack overflow.
当试图回答这样的问题时,您确实需要给出您提出的解决方案代码的局限性。如果只是关于性能,我不会太介意,但是作为解决方案提出的大多数代码(包括接受的答案)都无法压平任何深度大于 1000 的列表。
当我说大多数代码 我的意思是所有使用任何形式的递归的代码(或调用递归的标准库函数)。所有这些代码都会失败,因为对于每一次递归调用,(调用)堆栈都会增长一个单位,并且(默认)Python 调用堆栈的大小为 1000。
如果您对调用堆栈不太熟悉,那么也许以下内容会有所帮助(否则您可以滚动到实现)。
调用堆栈大小和递归编程(地牢类比)
寻找宝藏并退出
想象一下,您进入一个带有编号房间的巨大地牢,正在寻找宝藏。你不知道这个地方,但你有一些关于如何找到宝藏的指示。每个指示都是一个谜语(难度各不相同,但你无法预测它们有多难)。您决定考虑一下节省时间的策略,您做出了两个观察:
进入地牢时,你会注意到这里有一个小笔记本。你决定用它来记下解开谜语后(进入新房间时)退出的每个房间,这样你就可以返回入口。这是一个天才的想法,您甚至不会花一分钱来实施您的策略。
你进入了地牢,并成功解开了前 1001 个谜语,但出现了一些你没有计划的事情,你借来的笔记本里已经没有空间了。您决定放弃您的任务,因为您宁愿没有宝藏也不愿永远迷失在地牢中(这看起来确实很聪明)。
执行递归程序
基本上,这与寻找宝藏完全相同。地牢是计算机的内存,你现在的目标不是找到宝藏,而是计算一些函数(找到f(x)给定x)。这些指示只是帮助您求解 f(x) 的子例程。你的策略与调用堆栈策略相同,笔记本是堆栈,房间是函数的返回地址:
你在地牢中遇到的问题在这里也是一样的,调用堆栈有有限的大小(此处为 1000),因此,如果您输入太多函数而不返回,那么您将填满调用堆栈并出现类似
的错误“亲爱的冒险家,我很抱歉,但您的笔记本是full":RecursionError:超出最大递归深度
。请注意,您不需要递归来填充调用堆栈,但非递归程序不太可能调用 1000 个函数而不返回。还需要了解的是,一旦从函数返回,调用堆栈就会从所使用的地址中释放(因此名称为“堆栈”,返回地址在进入函数之前被压入,在返回时被拉出)。在简单递归的特殊情况下(函数f
调用自身一次——一遍又一遍——),您将一遍又一遍地输入f
直到计算完成(直到找到宝藏)并从f
返回,直到回到最初调用f
的地方。调用堆栈永远不会从任何内容中释放,直到最后它会一个接一个地从所有返回地址中释放。如何避免这个问题?
这实际上非常简单:“如果您不知道递归可以有多深,就不要使用递归”。这并不总是正确的,因为在某些情况下,可以优化尾调用递归 (TCO) 。但在Python中,情况并非如此,即使是“写得好的”递归函数也不会优化堆栈的使用。 Guido 有一篇关于这个问题的有趣文章: 尾递归消除。
您可以使用一种技术来使任何递归函数迭代,我们可以将这种技术称为自带笔记本。例如,在我们的特定情况下,我们只是探索一个列表,进入一个房间相当于进入一个子列表,您应该问自己的问题是如何从列表返回到其父列表? 答案并不复杂,重复以下操作,直到
stack
为空:address
和index
压入stack
当进入一个新的子列表时(注意列表地址+索引也是一个地址,因此我们只使用与调用堆栈使用的完全相同的技术);yield
它(或将它们添加到列表中);堆栈
返回父列表返回地址
(和索引
) 。另请注意,这相当于树中的 DFS,其中一些节点是子列表
A = [1, 2]
,一些是简单项:0, 1, 2, 3, 4< /code> (对于
L = [0, [1,2], 3, 4]
)。该树如下所示:DFS 遍历前序为:L、0、A、1、2、3、4。请记住,为了实现迭代 DFS,您还“需要”堆栈。我之前提出的实现会导致以下状态(对于
stack
和flat_list
):在本例中,堆栈最大大小为 2,因为输入列表 (因此树)的深度为 2。
实现
对于实现,在 python 中,您可以通过使用迭代器而不是简单的列表来简化一点。对(子)迭代器的引用将用于存储子列表返回地址(而不是同时具有列表地址和索引)。这并不是一个很大的区别,但我觉得这更具可读性(而且也更快一点):
另外,请注意,在
is_list_like
中,我有isinstance(item, list)
,可以更改它以处理更多输入类型,在这里我只想拥有最简单的版本,其中 (iterable) 只是一个列表。但您也可以这样做:这将字符串视为“简单项”,因此
flatten_iter([["test", "a"], "b])
将返回["test" , "a", "b"]
而不是["t", "e", "s", "t", "a", "b"]
。 上被调用两次,让我们假设这是读者的一个练习,以使其更简洁。在这种情况下,
iter(item)
在每个项目最后,请记住,您可以'。 t 使用
print(L)
打印无限嵌套列表L
因为在内部它将使用对__repr__
的递归调用 (RecursionError: max recursion 相同的错误消息。
出于同样的原因,涉及
str
的flatten
解决方案将失败,并显示 要测试您的解决方案,您可以使用此函数生成一个简单的嵌套列表:其中给出:
build_deep_list(5)
>>>[4, [3, [2, [ 1、[0]]]]]
。When trying to answer such a question you really need to give the limitations of the code you propose as a solution. If it was only about performances I wouldn't mind too much, but most of the codes proposed as solution (including the accepted answer) fail to flatten any list that has a depth greater than 1000.
When I say most of the codes I mean all codes that use any form of recursion (or call a standard library function that is recursive). All these codes fail because for every of the recursive call made, the (call) stack grow by one unit, and the (default) python call stack has a size of 1000.
If you're not too familiar with the call stack, then maybe the following will help (otherwise you can just scroll to the Implementation).
Call stack size and recursive programming (dungeon analogy)
Finding the treasure and exit
Imagine you enter a huge dungeon with numbered rooms, looking for a treasure. You don't know the place but you have some indications on how to find the treasure. Each indication is a riddle (difficulty varies, but you can't predict how hard they will be). You decide to think a little bit about a strategy to save time, you make two observations:
When entering the dungeon, you notice a small notebook here. You decide to use it to write down every room you exit after solving a riddle (when entering a new room), this way you'll be able to return back to the entrance. That's a genius idea, you won't even spend a cent implementing your strategy.
You enter the dungeon, solving with great success the first 1001 riddles, but here comes something you hadn't planed, you have no space left in the notebook you borrowed. You decide to abandon your quest as you prefer not having the treasure than being lost forever inside the dungeon (that looks smart indeed).
Executing a recursive program
Basically, it's the exact same thing as finding the treasure. The dungeon is the computer's memory, your goal now is not to find a treasure but to compute some function (find f(x) for a given x). The indications simply are sub-routines that will help you solving f(x). Your strategy is the same as the call stack strategy, the notebook is the stack, the rooms are the functions' return addresses:
The problem you encountered in the dungeon will be the same here, the call stack has a finite size (here 1000) and therefore, if you enter too many functions without returning back then you'll fill the call stack and have an error that look like
"Dear adventurer, I'm very sorry but your notebook is full":RecursionError: maximum recursion depth exceeded
. Note that you don't need recursion to fill the call stack, but it's very unlikely that a non-recursive program call 1000 functions without ever returning. It's important to also understand that once you returned from a function, the call stack is freed from the address used (hence the name "stack", return address are pushed in before entering a function and pulled out when returning). In the special case of a simple recursion (a functionf
that call itself once -- over and over --) you will enterf
over and over until the computation is finished (until the treasure is found) and return fromf
until you go back to the place where you calledf
in the first place. The call stack will never be freed from anything until the end where it will be freed from all return addresses one after the other.How to avoid this issue?
That's actually pretty simple: "don't use recursion if you don't know how deep it can go". That's not always true as in some cases, Tail Call recursion can be Optimized (TCO). But in python, this is not the case, and even "well written" recursive function will not optimize stack use. There is an interesting post from Guido about this question: Tail Recursion Elimination.
There is a technique that you can use to make any recursive function iterative, this technique we could call bring your own notebook. For example, in our particular case we simply are exploring a list, entering a room is equivalent to entering a sublist, the question you should ask yourself is how can I get back from a list to its parent list? The answer is not that complex, repeat the following until the
stack
is empty:address
andindex
in astack
when entering a new sublist (note that a list address+index is also an address, therefore we just use the exact same technique used by the call stack);yield
it (or add them in a list);stack
returnaddress
(andindex
).Also note that this is equivalent to a DFS in a tree where some nodes are sublists
A = [1, 2]
and some are simple items:0, 1, 2, 3, 4
(forL = [0, [1,2], 3, 4]
). The tree looks like this:The DFS traversal pre-order is: L, 0, A, 1, 2, 3, 4. Remember, in order to implement an iterative DFS you also "need" a stack. The implementation I proposed before result in having the following states (for the
stack
and theflat_list
):In this example, the stack maximum size is 2, because the input list (and therefore the tree) have depth 2.
Implementation
For the implementation, in python you can simplify a little bit by using iterators instead of simple lists. References to the (sub)iterators will be used to store sublists return addresses (instead of having both the list address and the index). This is not a big difference but I feel this is more readable (and also a bit faster):
Also, notice that in
is_list_like
I haveisinstance(item, list)
, which could be changed to handle more input types, here I just wanted to have the simplest version where (iterable) is just a list. But you could also do that:This considers strings as "simple items" and therefore
flatten_iter([["test", "a"], "b])
will return["test", "a", "b"]
and not["t", "e", "s", "t", "a", "b"]
. Remark that in that case,iter(item)
is called twice on each item, let's pretend it's an exercise for the reader to make this cleaner.Testing and remarks on other implementations
In the end, remember that you can't print a infinitely nested list
L
usingprint(L)
because internally it will use recursive calls to__repr__
(RecursionError: maximum recursion depth exceeded while getting the repr of an object
). For the same reason, solutions toflatten
involvingstr
will fail with the same error message.If you need to test your solution, you can use this function to generate a simple nested list:
Which gives:
build_deep_list(5)
>>>[4, [3, [2, [1, [0]]]]]
.尽管已经选择了一个优雅且非常Pythonic的答案,但我将提出我的解决方案以供审查:
请告诉我这段代码有多好或多坏?
Although an elegant and very pythonic answer has been selected I would present my solution just for the review:
Please tell how good or bad this code is?
我更喜欢简单的答案。没有发电机。没有递归或递归限制。只是迭代:
这适用于两个列表:内部 for 循环和外部 while 循环。
内部 for 循环遍历列表。如果它找到一个列表元素,它 (1) 使用 list.extend() 将该部分展平为一级嵌套,并且 (2) 将 keepChecking 切换为 True。 keepchecking 用于控制外层 while 循环。如果外循环设置为 true,则会触发内循环进行另一遍。
这些传递不断发生,直到找不到更多嵌套列表为止。当最终在没有找到任何内容的情况下发生传递时,keepChecking 永远不会跳到 true,这意味着 listIsNested 保持 false 并且外部 while 循环退出。
然后返回展平的列表。
测试运行
[1, 2, 3, 4, 100, 200, 300, 1000, 2000, 3000]
I prefer simple answers. No generators. No recursion or recursion limits. Just iteration:
This works with two lists: an inner for loop and an outer while loop.
The inner for loop iterates through the list. If it finds a list element, it (1) uses list.extend() to flatten that part one level of nesting and (2) switches keepChecking to True. keepchecking is used to control the outer while loop. If the outer loop gets set to true, it triggers the inner loop for another pass.
Those passes keep happening until no more nested lists are found. When a pass finally occurs where none are found, keepChecking never gets tripped to true, which means listIsNested stays false and the outer while loop exits.
The flattened list is then returned.
Test-run
[1, 2, 3, 4, 100, 200, 300, 1000, 2000, 3000]
我没有在这里查看所有已经可用的答案,但这是我想出的一个行,借用 lisp 的第一个和其余列表处理方式,
这里是一个简单的和一个不那么简单的情况 -
I didn't go through all the already available answers here, but here is a one liner I came up with, borrowing from lisp's way of first and rest list processing
here is one simple and one not-so-simple case -
我经常使用
more_itertools.collapse
:它直接处理字符串/字节,并且不会将它们解包为单个字符:
它还可以在给定的嵌套级别停止:
完整的代码相对简单,并且还使用递归方法:
I often use
more_itertools.collapse
:It handles strings/bytes out-of-the-box and doesn't unpack them into single characters:
It can also stop at a given nesting level:
The full code is relatively straighforward and it also uses a recursive approach:
我不确定这是否一定更快或更有效,但这就是我所做的:
这里的
flatten
函数将列表转换为字符串,取出 all方括号,将方括号重新附加到两端,然后将其变回列表。不过,如果您知道字符串列表中会有方括号,例如
[[1, 2], "[3, 4] 和 [5]"]
,您将不得不做一些事情别的。I'm not sure if this is necessarily quicker or more effective, but this is what I do:
The
flatten
function here turns the list into a string, takes out all of the square brackets, attaches square brackets back onto the ends, and turns it back into a list.Although, if you knew you would have square brackets in your list in strings, like
[[1, 2], "[3, 4] and [5]"]
, you would have to do something else.只需使用
funcy
库:pip 安装功能
Just use a
funcy
library:pip install funcy
没有递归或嵌套循环。几行。格式良好且易于阅读:
No recursion or nested loops. A few lines. Well formatted and easy to read:
这是 2.7.5 中的
compiler.ast.flatten
实现:还有更好、更快的方法(如果您已经到达这里,您已经看到它们)
另请注意:
Here's the
compiler.ast.flatten
implementation in 2.7.5:There are better, faster methods (If you've reached here, you have seen them already)
Also note:
我很惊讶没有人想到这一点。该死的递归我没有得到这里的高级人员所做的递归答案。无论如何,这是我对此的尝试。需要注意的是,它非常特定于 OP 的用例
输出:
I'm surprised no one has thought of this. Damn recursion I don't get the recursive answers that the advanced people here made. anyway here is my attempt on this. caveat is it's very specific to the OP's use case
output:
我知道已经有很多很棒的答案,但我想添加一个使用函数式编程方法来解决问题的答案。在这个答案中,我使用双重递归:
输出:
I am aware that there are already many awesome answers but i wanted to add an answer that uses the functional programming method of solving the question. In this answer i make use of double recursion :
output:
我在这里没有看到类似的内容,只是从同一主题的封闭问题中得到这里,但为什么不做这样的事情(如果您知道要拆分的列表的类型):
您需要了解元素的类型,但我认为这可以概括,并且就速度而言,我认为它会更快。
I don't see anything like this posted around here and just got here from a closed question on the same subject, but why not just do something like this(if you know the type of the list you want to split):
You would need to know the type of the elements but I think this can be generalised and in terms of speed I think it would be faster.
完全hacky,但我认为它会起作用(取决于你的数据类型)
totally hacky but I think it would work (depending on your data_type)
这是另一种 py2 方法,我不确定它是否是最快或最优雅或最安全的...
它可以忽略您想要的任何特定(或派生)类型,它返回一个迭代器,因此您可以将其转换为任何特定容器例如列表、元组、字典,或者只是消耗它以减少内存占用,无论好坏,它都可以处理初始的不可迭代对象,例如 int ...
请注意,大部分繁重的工作都是在 C 中完成的,因为到目前为止据我所知,这就是 itertools 的实现方式,所以虽然它是递归的,但据我所知,它不受 python 递归深度的限制,因为函数调用发生在 C 中,尽管这并不意味着你受到内存的限制,特别是在 OS X 中截至目前,其堆栈大小有硬性限制(OS X Mavericks)...
有一种稍微快一点的方法,但可移植性较差,仅当您可以假设可以明确确定输入的基本元素时才使用它,你会得到一个无限递归,而 OS X 的堆栈大小有限,会很快抛出分段错误......
这里我们使用集合来检查类型,所以它需要 O(1) vs O(number of types) 来检查是否应忽略某个元素,当然,任何具有指定忽略类型的派生类型的值都会失败,这就是为什么它使用
str
,unicode
因此请谨慎使用...测试:
Here is another py2 approach, Im not sure if its the fastest or the most elegant nor safest ...
It can ignore any specific (or derived) type you would like, it returns an iterator, so you can convert it to any specific container such as list, tuple, dict or simply consume it in order to reduce memory footprint, for better or worse it can handle initial non-iterable objects such as int ...
Note most of the heavy lifting is done in C, since as far as I know thats how itertools are implemented, so while it is recursive, AFAIK it isn't bounded by python recursion depth since the function calls are happening in C, though this doesn't mean you are bounded by memory, specially in OS X where its stack size has a hard limit as of today (OS X Mavericks) ...
there is a slightly faster approach, but less portable method, only use it if you can assume that the base elements of the input can be explicitly determined otherwise, you'll get an infinite recursion, and OS X with its limited stack size, will throw a segmentation fault fairly quickly ...
here we are using sets to check for the type so it takes O(1) vs O(number of types) to check whether or not an element should be ignored, though of course any value with derived type of the stated ignored types will fail, this is why its using
str
,unicode
so use it with caution ...tests:
不使用任何库:
Without using any library:
使用itertools.chain:
或者不使用链接:
Using
itertools.chain
:Or without chaining:
我使用递归来解决任意深度的嵌套列表
所以在我定义函数combine_nlist之后,很容易使用这个函数进行扁平化。或者您可以将其合并为一个函数。我喜欢我的解决方案,因为它可以应用于任何嵌套列表。
结果
I used recursive to solve nested list with any depth
So after i define function combine_nlist, it is easy to use this function do flatting. Or you can combine it into one function. I like my solution because it can be applied to any nested list.
result
最简单的方法是使用
pip install morph
来使用 morph 库。代码是:
The easiest way is to use the morph library using
pip install morph
.The code is: