与 Python 3.1.x 相比,Python 3.2 非常慢
我通读了Python 3.2的变化,了解到它比3.1做了很多改进。然而,我在 3.2 上运行的零修改完全相同的代码比在 3.1.3 上运行代码慢了 10 倍以上
Python 3.2 花了六分钟将文件的二进制内容传输到物理设备然后接收并打印出来屏幕上接收到的数据,而在同一台 PC 上使用 Python 3.1.3 执行完全相同的场景仅需 30 秒。
我使用 Python 3.1.2 从头开始开发代码,20% 的代码使用 ctypes 通过带有 USB/PCI 设备的 Windows 驱动程序执行事务,因此我认为这种性能影响与向后兼容性没有任何关系。在我的应用程序中,我创建了四个 threading.Thread 子类实例,每个实例处理系统上的一个 PCI 或 USB 设备。我怀疑 3.2 的 ctypes 性能比以往任何时候都更差,或者我必须使用更多的 threading.Thread 才能获得我想要的多线程性能。如果有人能为我遮挡一些灯光,我将不胜感激
========================================== ===
更具诊断性
我减少了要发送和接收的数据量
python 3.1.3 花费 3 秒来完成,如系统资源监视器屏幕截图所示 http://img62.imageshack.us/img62/5313/python313.png
python 3.2 花费大约 1 分钟才能完成,如系统资源监视器屏幕截图所示 http://img197.imageshack.us/img197/8366/python32.png
我的电脑是具有 2 GB RAM 的单核 Intel P4,因此我认为我们可以排除多核处理器的 GIL 因素。
我使用 yappi 来分析多次运行,以平均 3.1.3 和 3.2 上的性能结果。我发现线程和 ctypes 在 Python 3.2 上执行得很差。
这是访问由 python 包的标准 Windows 二进制文件提供的线程安全队列,
on 3.1.3
name #n tsub ttot tavg
C:\Python31\lib\queue.py.qsize:86 46070 1.352867 4.234082 0.000092
C:\Python31\lib\queue.py._get:225 8305 0.012457 0.017030 0.000002
C:\Python31\lib\queue.py.get:167 8305 0.635926 1.681601 0.000202
C:\Python31\lib\queue.py._put:221 8305 0.016156 0.020717 0.000002
C:\Python31\lib\queue.py.put:124 8305 0.095320 1.138560 0.000137
on 3.2
name #n tsub ttot tavg
C:\Python32\lib\queue.py.qsize:86 252168 4.987339 15.229308 0.000060
C:\Python32\lib\queue.py._get:225 8305 0.030431 0.035152 0.000004
C:\Python32\lib\queue.py.get:167 8305 0.303126 7.898754 0.000951
C:\Python32\lib\queue.py._put:221 8305 0.015728 0.020928 0.000003
C:\Python32\lib\queue.py.put:124 8305 0.143086 0.431970 0.000052
在 Python 3.2 另一个示例中,线程性能非常糟糕
。这个函数只是通过 ctypes 模块调用 Windows USB 驱动程序中的 API,并从 USB 设备请求 16 位数据
on 3.1.3
name #n tsub ttot tavg
..ckUSBInterface.py.read_register:14 1 0.000421 0.000431 0.000431
on 3.2
name #n tsub ttot tavg
..ckUSBInterface.py.read_register:14 1 0.015637 0.015651 0.015651
,如您所见,所花费的时间在 Python 3.2 上要差 30 倍以上
Python 3.2 对于我的应用程序来说似乎是一场灾难
I read through Python 3.2 changes and understand that it has made many improvement over 3.1. However, my exact same code with zero modification running on 3.2 is more than 10 times slower than when I run my code on 3.1.3
It took Python 3.2 six minutes to transfer binary content of a file to a physical device then receive and prints out the received data on screen, when the exact same scenario on same PC only takes 30 second to execute with Python 3.1.3.
I developed my code from scratch with Python 3.1.2 and 20% of my code uses ctypes to perform transaction through windows driver with USB/PCI device, so I don't think this performance hit has anything to do with backward compatibility. In my application, I create four instances of threading.Thread subclasses, each dealing with one PCI or USB device on the system. Things I suspect are that ctypes performance of 3.2 got worse than ever or there are more to threading.Thread that I have to play with to get exactly the multi-threading performance I want. Would be much appreciated if anyone can shade some lights for me
=========================================
more diagopnistic
I reduced amount of data to be sent&received
python 3.1.3 spends 3 seconds to comelete as shown in this system resource monitor screenshot http://img62.imageshack.us/img62/5313/python313.png
python 3.2 spends about 1 minutes to complete as shown in this system resource monitor screenshot http://img197.imageshack.us/img197/8366/python32.png
My PC is a single core Intel P4 with 2 GB of RAM, so I think we can rule out GIL factor for multiple core processors.
I used yappi to profile multiple runs to average out performance results on both 3.1.3 and 3.2. I see that threading and ctypes are badly performed on Python 3.2.
This is accessing thread safe queue provided with standard windows binary of python package
on 3.1.3
name #n tsub ttot tavg
C:\Python31\lib\queue.py.qsize:86 46070 1.352867 4.234082 0.000092
C:\Python31\lib\queue.py._get:225 8305 0.012457 0.017030 0.000002
C:\Python31\lib\queue.py.get:167 8305 0.635926 1.681601 0.000202
C:\Python31\lib\queue.py._put:221 8305 0.016156 0.020717 0.000002
C:\Python31\lib\queue.py.put:124 8305 0.095320 1.138560 0.000137
on 3.2
name #n tsub ttot tavg
C:\Python32\lib\queue.py.qsize:86 252168 4.987339 15.229308 0.000060
C:\Python32\lib\queue.py._get:225 8305 0.030431 0.035152 0.000004
C:\Python32\lib\queue.py.get:167 8305 0.303126 7.898754 0.000951
C:\Python32\lib\queue.py._put:221 8305 0.015728 0.020928 0.000003
C:\Python32\lib\queue.py.put:124 8305 0.143086 0.431970 0.000052
thread-wise performance is just insanely bad on Python 3.2
another example. this function simply calls API in windows USB driver through ctypes module and request 16 bits of data from USB device
on 3.1.3
name #n tsub ttot tavg
..ckUSBInterface.py.read_register:14 1 0.000421 0.000431 0.000431
on 3.2
name #n tsub ttot tavg
..ckUSBInterface.py.read_register:14 1 0.015637 0.015651 0.015651
as you can see, the time it takes is more than 30 times worse on Python 3.2
Python 3.2 seems like a disaster for my application
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
没有明显的理由说明为什么会这样。您需要对应用程序进行分析,以准确了解哪些内容需要额外的时间。
There is no obvious reason why this should be. You'll need to profile the app to see exactly what takes this additional time.