Python Cprofile总时间
我正在对我的代码进行一些分析,并试图尽可能地优化它。但是我注意到,特定功能的总累积时间有时不会累加。因此,我不确定哪种功能是罪魁祸首。
为了简化这一点,我在下面创建了一个简单的示例 - 它实际上没有做任何事情。
L = [1,2,3,4,5]
class MyObj:
__slots__ = "a", "b"
def __init__(self, a="", b=""):
self.a = a
self.b = b
def get_top():
return L[0]
def revert(x):
return -x
def check():
if True:
return True
else:
return False
def main(x):
if check():
p = revert(x)
q = get_top()
o = MyObj(a=p, b=q)
else:
o = MyObj()
for x in range(10_000_000):
main(x)
而且,如果我使用pypy3执行此操作(相同的情况在python3中确实发生了)代码:
python -m cProfile test.py
我得到类似的东西:
50000005 function calls in 1.895 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.522 0.522 1.895 1.895 test0.py:1(<module>)
10000000 0.180 0.000 0.180 0.000 test0.py:12(get_top)
10000000 0.153 0.000 0.153 0.000 test0.py:16(revert)
10000000 0.140 0.000 0.140 0.000 test0.py:20(check)
10000000 0.746 0.000 1.374 0.000 test0.py:27(main)
1 0.000 0.000 0.000 0.000 test0.py:4(MyObj)
10000000 0.156 0.000 0.156 0.000 test0.py:7(__init__)
1 0.000 0.000 0.000 0.000 {built-in function __build_class__}
1 0.000 0.000 1.895 1.895 {built-in function exec}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
如您所见,主花费了1.374 cumtime和0.746搅拌器,但是当添加调用的函数时,它将小于这个数字。这总是不会加起来吗?只是想知道我如何分辨要花费一段时间的功能是否准确 - 假设函数就像由多个函数组成的main()
。
I am doing some profiling of my code, and trying to optimize it as much as I could. But I notice that the total cumulative time for a specific function doesnt add up sometimes. So I wasnt sure which function was being the culprit.
To make this easier, I created a simple example below -- it doesnt do anything really.
L = [1,2,3,4,5]
class MyObj:
__slots__ = "a", "b"
def __init__(self, a="", b=""):
self.a = a
self.b = b
def get_top():
return L[0]
def revert(x):
return -x
def check():
if True:
return True
else:
return False
def main(x):
if check():
p = revert(x)
q = get_top()
o = MyObj(a=p, b=q)
else:
o = MyObj()
for x in range(10_000_000):
main(x)
And if I execute this with pypy3 (same scenario happens with python3 really) with code:
python -m cProfile test.py
I get something like:
50000005 function calls in 1.895 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.522 0.522 1.895 1.895 test0.py:1(<module>)
10000000 0.180 0.000 0.180 0.000 test0.py:12(get_top)
10000000 0.153 0.000 0.153 0.000 test0.py:16(revert)
10000000 0.140 0.000 0.140 0.000 test0.py:20(check)
10000000 0.746 0.000 1.374 0.000 test0.py:27(main)
1 0.000 0.000 0.000 0.000 test0.py:4(MyObj)
10000000 0.156 0.000 0.156 0.000 test0.py:7(__init__)
1 0.000 0.000 0.000 0.000 {built-in function __build_class__}
1 0.000 0.000 1.895 1.895 {built-in function exec}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
As you can see, the main took 1.374 cumtime and 0.746 tottime, but when adding up the functions called, it will be less than this number. Is this always not going to add up? Just wondering how I can tell the function that is taking a while if each function time is not accurate -- assuming the function is like the main()
which consists of multiple functions.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
有一些提示pypy 在这里。通常,像Cprofile这样的确定性剖析师会破坏JIT,因此不建议与PYPY进行分析。统计方法是,在每个X毫秒中堆叠程序堆叠程序的过程中,统计方法更好。使用JIT时,分析和优化是严重的问题,因为通常不会加起来:解释器本身中的开销很难衡量。
There are some hints for profiling PyPy here. In general, a deterministic profiler like cprofile will break the JIT, so it is not recommended for profiling with PyPy. A statistical approach is better where the profiler samples the program stack every X milliseconds. Profiling and optimization are hard problems when using a JIT because often things don't add up: there is overhead in the interpreter itself that are hard to measure.