关于生成器表达式和速度高效替代方案的几个问题
考虑以下代码,它是我下面的问题的组成部分:
import functools
N = 3
class Struct:
"""Create an instance with argument=value slots.
This is for making a lightweight object whose class doesn't matter."""
def __init__(self, **entries):
self.__dict__.update(entries)
def __repr__(self):
args = ['%s=%s' % (k, repr(v)) for (k, v) in vars(self).items()]
return '\nStruct(%s)' % ', '.join(args)
def doit( move ):
( rowIn, colIn ) = move
something = rowIn + ( 10 * colIn ) # An involved computation here in real life
return Struct( coord = ( rowIn, colIn ), something = something )
legalStates = [ ( row, col ) for row in xrange( N ) for col in xrange( N ) ] # A more complicated function that generates the list in real life. Call it 'complicatedFunction'
genExpFn = lambda : ( ( s.something, m, s ) for ( m, s ) in ( ( move, doit( move ) ) for move in legalStates ) ) #Q1
successorsSortedGenFn = lambda : ( p for p in sorted( genExpFn(), reverse = True ) )
def bFunc( s, a ):
#print "a * s ->", a * s
return a * s # An involved computation here in real life
def aFunc( ( v, m, s ) ): #Q2
assert( s.something == v )
return bFunc( s.something, 10 )
print "min( successorsSortedGen ) -> " + str( min( successorsSortedGenFn(), key=functools.partial( aFunc )) ) #Q3
print
print "max( successorsSortedGen ) -> " + str( max( successorsSortedGenFn(), key=functools.partial( aFunc )) ) #Q4
我的问题基于标记为“#Q”的语句:
Q1:很明显,生成器已完全实例化(所有元素均已执行),如下所示我们对其调用 sorted()
(它生成所有元素并创建一个临时未排序列表,对其进行排序并作为新列表返回?)。
是否有一种节省空间的方法,可以最大限度地减少临时对象的创建并产生排序列表?
我尝试过,但无法编写可以使用 list.sort()
就地排序的列表理解,
这是我正在考虑的表达式类型:
successorsSorted = [ ( s.something, m, s ) for ( m, s ) in ( ( move, doit( move ) ) for move in legalStates ) ].sort( reverse = True )
Q2:注意“aFunc”只是“bFunc”的包装,因为我无法在 functools.partial( aFunc ) 调用中编写等效的表示形式。
我正在寻找的 functools.partial( aFunc )
中允许我直接调用“bFunc”的表达式“aFunc”是什么??
编辑:第二个问题的答案是lambda ( v, m, s ): bFunc(s.something, 10)
因此,语句变成:
print "min( successorsSortedGen ) -> " + str( min( successorsSortedGenFn(), key=functools.partial( lambda ( v, m, s ): bFunc(s.something, 10)) ) )
print
print "max( successorsSortedGen ) -> " + str( max( successorsSortedGenFn(), key=functools.partial( lambda ( v, m, s ): bFunc(s.something, 10)) ) )
我知道这看起来有点蹩脚我之前没有考虑到这一点,但是哦,好吧(感谢 aaronasterling 的温和鼓励)。
Q3、Q4:请注意,传递给 min() 和 max() 的元素已经排序。
我是否可以对 min() 和 max() 做出此提示,以便它不会将整个列表实例化为临时列表,然后迭代整个列表以找到 min 或 max 元素?
如果没有,是否存在一个模块或自定义函数,它不会实例化整个列表,但是,考虑到传递给它的列表已排序,在检查最少元素数时返回最小或最大元素?
Consider the following code, integral to my questions below:
import functools
N = 3
class Struct:
"""Create an instance with argument=value slots.
This is for making a lightweight object whose class doesn't matter."""
def __init__(self, **entries):
self.__dict__.update(entries)
def __repr__(self):
args = ['%s=%s' % (k, repr(v)) for (k, v) in vars(self).items()]
return '\nStruct(%s)' % ', '.join(args)
def doit( move ):
( rowIn, colIn ) = move
something = rowIn + ( 10 * colIn ) # An involved computation here in real life
return Struct( coord = ( rowIn, colIn ), something = something )
legalStates = [ ( row, col ) for row in xrange( N ) for col in xrange( N ) ] # A more complicated function that generates the list in real life. Call it 'complicatedFunction'
genExpFn = lambda : ( ( s.something, m, s ) for ( m, s ) in ( ( move, doit( move ) ) for move in legalStates ) ) #Q1
successorsSortedGenFn = lambda : ( p for p in sorted( genExpFn(), reverse = True ) )
def bFunc( s, a ):
#print "a * s ->", a * s
return a * s # An involved computation here in real life
def aFunc( ( v, m, s ) ): #Q2
assert( s.something == v )
return bFunc( s.something, 10 )
print "min( successorsSortedGen ) -> " + str( min( successorsSortedGenFn(), key=functools.partial( aFunc )) ) #Q3
print
print "max( successorsSortedGen ) -> " + str( max( successorsSortedGenFn(), key=functools.partial( aFunc )) ) #Q4
My questions are based on the statements marked as "#Q":
Q1 : It's apparent that the generator is completely instantiated ( all elements are executed ), as we call sorted()
on it ( which generates all the elements and creates a temporary unsorted list which it sorts and returns as a new list? ).
Is there a space efficient way, that minimizes the creation of temporaries and yeilds a sorted list?
I tried to, but could not write a list comprehension that could sort in place using list.sort()
This was the kind of expression I was thinking about:
successorsSorted = [ ( s.something, m, s ) for ( m, s ) in ( ( move, doit( move ) ) for move in legalStates ) ].sort( reverse = True )
Q2 : Notice that 'aFunc' is just a wrapper around 'bFunc' because I was not able to write an equivalent representation in the functools.partial( aFunc )
call.
What is the expression 'aFunc' in functools.partial( aFunc )
that I am looking for that will allow me to call 'bFunc' directly?
EDIT : The answer to Q2 is lambda ( v, m, s ): bFunc(s.something, 10)
Thus, the statements become:
print "min( successorsSortedGen ) -> " + str( min( successorsSortedGenFn(), key=functools.partial( lambda ( v, m, s ): bFunc(s.something, 10)) ) )
print
print "max( successorsSortedGen ) -> " + str( max( successorsSortedGenFn(), key=functools.partial( lambda ( v, m, s ): bFunc(s.something, 10)) ) )
I know it kinda seems lame I did not think about this earlier, but oh well ( thanks to aaronasterling for the gentle prodding ).
Q3, Q4 : Note that the elements passed to min() and max() are already sorted.
Is it possible for me to make this hint to min() and max() so that it does not instantiate the whole list as a temporary and then iterate through the whole list to locate the min or max element?
If not, is there a module or custom function in existence that does not instantiate the whole list, but, given that the list passed to it is sorted, returns the min or max element while inspecting the least number of elements?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
Q1.
[x for x in somelist].sort()
创建一个列表并调用sort
方法。这将返回None
该赋值将None
分配给successorSorted
。如果你想这样做,你必须自己实现它,它可能会比创建临时列表的内置排序慢得多。Q2。您可以拆开代码对象并重新排列参数列表,使
a
成为第一个参数,然后重写所有字节码以考虑局部变量的新位置。 (是的,这实际上可以做到)。然后你可以使用 functools.partial 。或者您可以像现在一样或以其他一些方式使用包装器。我对包装纸+1。 (尽管如果您想要字节码破解,请告诉我,我认为它们很有趣,并且在 Stack Overflow 上提供它们作为答案的最酷的事情是我可以编写它们但不必使用它们;)Q3,Q4 。并不真地。要获取迭代器的第十个元素,您需要遍历所有前面的元素。如果你知道你想要第一个元素,你可以这样做
,对于最后
一个元素,第一个元素将吃掉迭代器的第一个元素,最后一个元素将吃掉整个迭代器。再见迭代器。
Q1.
[x for x in somelist].sort()
creates a list and calls thesort
method. This returnsNone
The assignment assignsNone
tosuccessorSorted
. If you want to do this, you'll have to implement it yourself and it will probably be way slower than the builtin sort creating the temporary list.Q2. You could take apart the code object and rearrange the argument list so that
a
is the first argument and then rewrite all the bytecode to account for the new positions of the locals. (Yes this can actually be done). You could then usefunctools.partial
on that. Or you could use a wrapper like you're doing now or in a few other ways. I'm +1 on the wrapper. (though let me know if you want a bytecode hack, I think they're fun and the cool thing about providing them as answers on Stack Overflow is that I get to write them but don't have to use them ;)Q3, Q4. Not really. To get the tenth element of an iterator, you need to go through all of the earlier ones. If you know that you want the first element, you can just do
and for the last
The first will eat the first element of the iterator and the last will eat your whole iterator. Bye Bye iterator.