如何在 Python 中使用捆绑/打包生成器？

发布于 2024-10-29 02:43:04 字数 420 浏览 4 评论 0原文

为了分摊函数调用开销，我更改了生成器，以便它生成一个包含几个值的固定长度列表，而不是一次生成一个值。生成器按照最初的样子，从包含多个 pickle 对象的文件中 unpickle 一个对象，并生成它。然后，这些内容在消耗生成器的 for 循环内进行处理。事实证明，这种方法比将对象处理代码放在手动展开的循环中（一次解封文件中的多个连续项目）要慢得多。我正在尝试妥协。我修改过的生成器一次生成一个固定长度的腌制对象列表。我正在寻找一种 Pythonic 方式来在消费者端解压未腌制的对象包。

有没有办法在没有额外嵌套循环的情况下解构这些数据包？我错误地认为 * 运算符会这样做：

for x in *packetizing_generator(): f(x)

嵌套循环当然可以工作，但我想知道是否有更短、更优雅的方法。

原文

To amortize the function call overhead I have changed my generator so that it yields a fixed-length list of a few values instead of yielding one value at a time. The generator, as it originally stood, unpickled an object from a file that contains several pickled objects, and yielded it. These were then processed inside a for loop that consumed the generator. This approach turned out to be a lot slower than having the object processing code inside a hand unrolled loop that unpickled several consecutive items in the file at a time. I am attempting a compromise. My modified generator yields a fixed-length list of pickled objects at a time. I am looking for a Pythonic way to unpack that packet of unpickled objects at the consumer side.

Is there a way to deconstruct those packets without having an extra nested loop ? I incorrectly assumed that the * operator will do it like so:

for x in *packetizing_generator(): f(x)

Nested loop of course works, but am wondering if there is a shorter and more elegant way.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

糖果控 2024-11-05 02:43:04

我想这就是你所说的，如果我不正确，请告诉我。

for a, b, c in your_generator:
    # do stuff

假设 your_generator 生成的结果恰好是 3 个项目的序列。

您可以使用以下方法构建一个一次恰好生成 n 个项目的生成器，以便在 for 循环中使用：

itertools.izip_longest(*[your_generator]*n)

如果您的生成器在某个时间没有生成单个项目，时间，并且您想要更改它生成的项目数，请使用以下命令：

itertools.izip_longest(*[itertools.chain(*your_generator)]*n)

这是一个示例：

>>> from itertools import izip_longest, chain, combinations
>>> for a, b, c, d in izip_longest(*[chain(*combinations(range(4), 2))]*4):
...     print a, b, c, d
... 
0 1 0 2
0 3 1 2
1 3 2 3

这需要一个最初一次生成两个项目的生成器，而是一次生成四个元素。

I think this is what you are talking about, let me know if I'm not on the right track.

for a, b, c in your_generator:
    # do stuff

Assumes your_generator yields results as a sequence of exactly 3 items.

You can construct a generator that yields exactly n items at a time to use in your for loop with the following method:

itertools.izip_longest(*[your_generator]*n)

And if your generator does not yield a single item at a time, and you want to change the number of items it yields you use the following:

itertools.izip_longest(*[itertools.chain(*your_generator)]*n)

Here is an example:

>>> from itertools import izip_longest, chain, combinations
>>> for a, b, c, d in izip_longest(*[chain(*combinations(range(4), 2))]*4):
...     print a, b, c, d
... 
0 1 0 2
0 3 1 2
1 3 2 3

This takes a generator that originally yielded items two at a time, and instead yields elements four at a time.

回复收藏 0 原文

~没有更多了~