生成器在输入迭代器的开头和结尾保留占位符完好无损

发布于 2024-12-09 19:24:48 字数 3406 浏览 1 评论 0原文

我们以一个列表为例：

a = [255, 255, 1, 255, 255, 255, 1, 2, 255, 255, 2, 255, 255, 3, 255, 3, 255, 255, 255]

255是其中的一个特殊值。这是一个占位符。

我制作了一个生成器来替换列表中的一些占位符。它按预期工作。

但我不需要处理开始占位符 [255, 255 和结束占位符 255, 255, 255] 并完整地生成它们。

因此，我尝试修改生成器来解决这个问题：

Python 2.7

from __future__ import print_function
from  itertools import tee, izip, ifilterfalse

def replace(iterable,placeholder=255):
    it = enumerate(iterable) #the position is needed for the logic for the middle of the list
    it = ifilterfalse(lambda x: x[1]==placeholder, it) #create an iterator that deletes all the placeholders
    for i,(left,right) in enumerate(window(it,2)): #Slide through the filtered list with the window of 2 elements
        if i==0: #Leaving the beginning placeholders intact
            for j in range(left[0]):
                yield placeholder

        #SOME LOGIC FOR THE MIDDLE OF THE LIST (it works well)

    #Need to leave the trailing placeholders intact.

转换为列表的临时值只是为了便于理解代码：

>>>iterable
[255,1,255,255,1,255,255,255,2,2,255,255,255,2,2,3,255,255,255,3,255,255]

>>>it = enumerate(iterable)
[(0, 255), (1, 1), (2, 255), (3, 255), (4, 1), (5, 255), (6, 255), (7, 255), (8, 2), (9, 2), (10, 255), (11, 255), (12, 255), (13, 2), (14, 2), (15, 3), (16, 255), (17, 255), (18, 255), (19, 3), (20, 255), (21, 255)]

>>>it = ifilterfalse(lambda x: x[1]==placeholder, it)
[(1, 1), (4, 1), (8, 2), (9, 2), (13, 2), (14, 2), (15, 3), (19, 3)]

>>>list(enumerate(window(it,2)))
[(0, ((1, 1), (4, 1))), (1, ((4, 1), (8, 2))), (2, ((8, 2), (9, 2))), (3, ((9, 2), (13, 2))), (4, ((13, 2), (14, 2))), (5, ((14, 2), (15, 3))), (6, ((15, 3), (19, 3)))]

因此，如您所见，list( enumerate(window(it,2))) 包含前导非占位符值的索引 (0, ((**1**, 1), (4, 1))),< /代码>，但事实并非如此包含初始迭代器有多少个尾随占位符的信息： list(enumerate(window(it,2))) 以此值结束 (6, ((15, 3), (* *19**, 3))) 它只有最后一个非占位符值的索引，它不提供剩余多少个占位符的信息。

我设法依靠 it = enumerate(iterable) 来处理前导占位符，它生成初始迭代器值的位置，该值保留在 ifilterfalse 生成的第一个值中。

但我花了很多时间试图弄清楚如何对尾随占位符做同样的事情。问题是 ifilterfalse 只是吞下了 enumerate(iterable) 的最后一个占位符值，我看不到如何访问它们（自第一次生成以来，前导占位符是可能的） ifilterfalse 的值包含 enumerate(iterable)) 值的索引。

问题

纠正此代码以处理尾随占位符的最佳方法是什么？

因为目标不是无论如何都要创建代码（我已经使用不同的技术完成了），我想通过修补来解决任务稍微修改一下代码，而不是完全重写它。

这更像是一种训练，而不是真正的任务。

其他信息

window 是来自这里。

我的代码与这个由@nye17回答。但在这段代码中，作者对初始列表进行了就地修改。我想创建一个生成器，它将产生与该代码中的结果列表相同的值。

此外，我希望我的生成器接受任何可迭代对象作为参数，而不仅仅是列表（例如，它可以接受从文件中一一读取值的迭代器）。仅使用列表作为参数，任务变得更简单，因为我们可以从末尾扫描列表。

这不是我生活中必须解决的真正任务。这只是为了一次训练。

完整代码 http://codepad.org/9UJ9comY

原文

Let's take a list as an example:

a = [255, 255, 1, 255, 255, 255, 1, 2, 255, 255, 2, 255, 255, 3, 255, 3, 255, 255, 255]

255 is a special value in it. It's a placeholder.

I've made a generator which replaces some of the placeholder inside the list. It works as expected.

But I need not to process the beginning placeholders [255, 255 and the ending placeholders 255, 255, 255] and yield them intact.

So, I tried to modify the generator to work it out:

Python 2.7

from __future__ import print_function
from  itertools import tee, izip, ifilterfalse

def replace(iterable,placeholder=255):
    it = enumerate(iterable) #the position is needed for the logic for the middle of the list
    it = ifilterfalse(lambda x: x[1]==placeholder, it) #create an iterator that deletes all the placeholders
    for i,(left,right) in enumerate(window(it,2)): #Slide through the filtered list with the window of 2 elements
        if i==0: #Leaving the beginning placeholders intact
            for j in range(left[0]):
                yield placeholder

        #SOME LOGIC FOR THE MIDDLE OF THE LIST (it works well)

    #Need to leave the trailing placeholders intact.

The interim values converted to list just to ease the comprehension of the code:

>>>iterable
[255,1,255,255,1,255,255,255,2,2,255,255,255,2,2,3,255,255,255,3,255,255]

>>>it = enumerate(iterable)
[(0, 255), (1, 1), (2, 255), (3, 255), (4, 1), (5, 255), (6, 255), (7, 255), (8, 2), (9, 2), (10, 255), (11, 255), (12, 255), (13, 2), (14, 2), (15, 3), (16, 255), (17, 255), (18, 255), (19, 3), (20, 255), (21, 255)]

>>>it = ifilterfalse(lambda x: x[1]==placeholder, it)
[(1, 1), (4, 1), (8, 2), (9, 2), (13, 2), (14, 2), (15, 3), (19, 3)]

>>>list(enumerate(window(it,2)))
[(0, ((1, 1), (4, 1))), (1, ((4, 1), (8, 2))), (2, ((8, 2), (9, 2))), (3, ((9, 2), (13, 2))), (4, ((13, 2), (14, 2))), (5, ((14, 2), (15, 3))), (6, ((15, 3), (19, 3)))]

So, as you can see, the list(enumerate(window(it,2))) contains the index of the leading non-placeholder value (0, ((**1**, 1), (4, 1))),, but it doesn't contain the information how many trailing placeholder the initial iterator had: list(enumerate(window(it,2))) ends in this value (6, ((15, 3), (**19**, 3))) which has only the index of the last non-placeholder value, which doesn't give the information how many placeholders are left.

I managed to process the leading placeholders by relying on it = enumerate(iterable) which yields the position of the initial iterator value which persists in the first yielded value by ifilterfalse.

But I spent quite a lot of time trying to figure out how to do the same thing with the trailing placeholders. The problem is that ifilterfalse just swallows the last placeholder values of enumerate(iterable) and I see no way to access them (it was possible for the leading placeholders since the first generated value of ifilterfalse contained the index of the value of the enumerate(iterable)).

Question

What is the best way to correct this code for it to process the trailing placeholders?

As the goal is not to create a code by all means (I have already done it using a different technique), I want to solve the task by tinkering a bit wit the code, not a complete rewriting it.

It's more of a training than a real task.

Additional information

window is the code from here.

My code does nearly the same as in this answer by @nye17. But in this code the author make inplace modifications of the initial list. And I want to create a generator which will be yielding the same values as the resultant list in that code.

Furthermore, I want my generator to accept any iterables as a parameter, not only lists (for example it may accept the iterator which reads the values from file one by one). With having only lists as a parameter, the task becomes simpler, since we can scan the list from the end.

This is not a real task I have to solve in life. It's just for a training.

Full code
http://codepad.org/9UJ9comY

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

残疾 2024-12-16 19:24:48

def replace(it, process, placeholder):
    it = iter(it)
    while True:
        item = it.next()
        if item == placeholder:
            yield item
        else:
            yield process(item)
    pcount = 0
    try:
        while True:
            item = it.next()
            if item == placeholder:
                pcount += 1
            else:
                for i in range(pcount):
                    yield process(placeholder)
                pcount = 0
                yield process(item)
    except StopIteration:
        for i in range(pcount):
            yield placeholder

像这样使用它：

>>> a = [0, 0, 1, 0, 0, 0, 1, 2, 0, 0, 2, 0, 0, 3, 0, 3, 0, 0, 0]
>>> [x for x in replace(a, lambda n: n+20, 0)]
[0, 0, 21, 20, 20, 20, 21, 22, 20, 20, 22, 20, 20, 23, 20, 23, 0, 0, 0]

def replace(it, process, placeholder):
    it = iter(it)
    while True:
        item = it.next()
        if item == placeholder:
            yield item
        else:
            yield process(item)
    pcount = 0
    try:
        while True:
            item = it.next()
            if item == placeholder:
                pcount += 1
            else:
                for i in range(pcount):
                    yield process(placeholder)
                pcount = 0
                yield process(item)
    except StopIteration:
        for i in range(pcount):
            yield placeholder

Use it like this:

>>> a = [0, 0, 1, 0, 0, 0, 1, 2, 0, 0, 2, 0, 0, 3, 0, 3, 0, 0, 0]
>>> [x for x in replace(a, lambda n: n+20, 0)]
[0, 0, 21, 20, 20, 20, 21, 22, 20, 20, 22, 20, 20, 23, 20, 23, 0, 0, 0]

回复收藏 0 原文

美羊羊 2024-12-16 19:24:48

def replace(it, placeholder):
    while True:
        curr = it.next()
        if curr == placeholder:
            yield curr
        else:
            break

    yield curr

    try:
        cache = []
        while True:      
            curr = it.next()

            if curr == placeholder:
                cache.append(curr)
            else:
                for cached in cache:
                    yield TRANSFORM(cached)
                yield curr
                cache = []

    except StopIteration:
        for cached in cache:
            yield cache

def replace(it, placeholder):
    while True:
        curr = it.next()
        if curr == placeholder:
            yield curr
        else:
            break

    yield curr

    try:
        cache = []
        while True:      
            curr = it.next()

            if curr == placeholder:
                cache.append(curr)
            else:
                for cached in cache:
                    yield TRANSFORM(cached)
                yield curr
                cache = []

    except StopIteration:
        for cached in cache:
            yield cache

回复收藏 0 原文

月亮坠入山谷 2024-12-16 19:24:48

我想出的最简单的解决方案是通过另一个生成器处理 it = enumerate(iterable) ，该生成器仅保存最后返回的值。

因此，我在 it = enumerate(iterable) 之后添加了以下代码（在 replace 函数内）：

def save_last(iterable):
        for i in iterable:
            yield i
        replace.last_index = i[0] #Save the last value
it = save_last(it)

在 iterable 耗尽后，最后一个运算符生成器的保存生成值的索引（即 i[0]，因为 enumerate 将其存储在元组的 0 位置）为这replace 属性（因为 replace 函数是类的实例，可以具有局部变量）。

it 包装在新创建的生成器 save_last 中。

在函数的最后，我添加了使用 replace.last_index 变量中保存的索引的代码。

if right[0]<replace.last_index:
    for i in range(replace.last_index-right[0]):
        yield placeholder

完整代码：

from __future__ import print_function
from  itertools import tee, izip, ifilterfalse


def window(iterable,n):
    els = tee(iterable,n)
    for i,el in enumerate(els):
        for _ in range(i):
            next(el, None)
    return izip(*els)


def replace(iterable,placeholder=255):
    it = enumerate(iterable)

    def save_last(iterable):
        for i in iterable:
            yield i
        replace.last_index = i[0] #Save the last value
    it = save_last(it)

    it = ifilterfalse(lambda x: x[1]==placeholder, it)
    for i,(left,right) in enumerate(window(it,2)):
        if i==0:
            for j in range(left[0]):
                yield placeholder
        yield left[1]
        if right[0]>left[0]+1:
            if left[1]==right[1]:
                for _ in range(right[0]-left[0]-1):
                    yield left[1]
            else:
                for _ in range(right[0]-left[0]-1):
                    yield placeholder
    yield right[1]
    if right[0]<replace.last_index:
        for i in range(replace.last_index-right[0]):
            yield placeholder


a = [255,1,255,255,1,255,255,255,2,2,255,255,255,2,2,3,255,255,255,3,255,255]        
print('\nInput: {}'.format(a))
output = list(replace(a))
print('Proram output: {}'.format(output))
print('Goal output  : {}'.format([255,1,1,1,1,255,255,255,2,2,2,2,2,2,2,3,3,3,3,3,255,255]))

按预期工作：

Input: [255, 1, 255, 255, 1, 255, 255, 255, 2, 2, 255, 255, 255, 2, 2, 3, 255, 255, 255, 3, 255, 255]
Proram output: [255, 1, 1, 1, 1, 255, 255, 255, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 255, 255]
Goal output  : [255, 1, 1, 1, 1, 255, 255, 255, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 255, 255]

我唯一不喜欢的是用 C 编写的非常高效的 ifilterfalse 和 save_last 的组合> 用 Python 编写。

The simplest solution I came up with is to process it = enumerate(iterable) through one more generator which just saves the last returned value by it.

So, I added the following code after it = enumerate(iterable) (inside the replace function):

def save_last(iterable):
        for i in iterable:
            yield i
        replace.last_index = i[0] #Save the last value
it = save_last(it)

After iterable is exhausted, the last operator of the generator saves the index of the yielded value (which is i[0] as enumerate stores it at the position 0 of tupele) as the replace attribute (since replace function is a instance of a class, which can have local variables).

The it is wrapped in the newly created generator save_last.

At the very end of the function I added the code which uses the saved index in replace.last_index variable.

if right[0]<replace.last_index:
    for i in range(replace.last_index-right[0]):
        yield placeholder

The full code:

from __future__ import print_function
from  itertools import tee, izip, ifilterfalse


def window(iterable,n):
    els = tee(iterable,n)
    for i,el in enumerate(els):
        for _ in range(i):
            next(el, None)
    return izip(*els)


def replace(iterable,placeholder=255):
    it = enumerate(iterable)

    def save_last(iterable):
        for i in iterable:
            yield i
        replace.last_index = i[0] #Save the last value
    it = save_last(it)

    it = ifilterfalse(lambda x: x[1]==placeholder, it)
    for i,(left,right) in enumerate(window(it,2)):
        if i==0:
            for j in range(left[0]):
                yield placeholder
        yield left[1]
        if right[0]>left[0]+1:
            if left[1]==right[1]:
                for _ in range(right[0]-left[0]-1):
                    yield left[1]
            else:
                for _ in range(right[0]-left[0]-1):
                    yield placeholder
    yield right[1]
    if right[0]<replace.last_index:
        for i in range(replace.last_index-right[0]):
            yield placeholder


a = [255,1,255,255,1,255,255,255,2,2,255,255,255,2,2,3,255,255,255,3,255,255]        
print('\nInput: {}'.format(a))
output = list(replace(a))
print('Proram output: {}'.format(output))
print('Goal output  : {}'.format([255,1,1,1,1,255,255,255,2,2,2,2,2,2,2,3,3,3,3,3,255,255]))

Which works as expected:

Input: [255, 1, 255, 255, 1, 255, 255, 255, 2, 2, 255, 255, 255, 2, 2, 3, 255, 255, 255, 3, 255, 255]
Proram output: [255, 1, 1, 1, 1, 255, 255, 255, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 255, 255]
Goal output  : [255, 1, 1, 1, 1, 255, 255, 255, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 255, 255]

The only thing that I don't like is the combination of very efficient written in C ifilterfalse and save_last written in Python.

回复收藏 0 原文

~没有更多了~