Stackless python网络性能随着时间的推移而下降?
因此,我正在尝试使用 stackless python,编写一个非常简单的网络服务器来教自己使用微线程/tasklet 进行编程。 但现在我的问题是,当我在 apache bench 中运行类似 ab -n 100000 -c 50 http://192.168.0.192/
(100k 请求,50 并发)时,我得到类似 6k req/ 的结果s,我第二次运行它时得到 5.5k,第三次得到 5k,第四次得到 4.5k,等等,一直下降到 100req/s 之类的。
不过,当我重新启动 python 脚本时,问题就消失了。
现在我的问题是为什么? 我忘记删除tasklet了吗? 我已经检查了 stackless.getruncount() (由于某种原因,它似乎总是返回 1),所以看起来不会有任何死亡的 tasklet 挂在周围? 我尝试在所有已完成的小任务上调用 .kill() ,但没有帮助。 我就是想不通这个问题。
import socket
import select
import stackless
import time
class socket_wrapper(object):
def __init__(self, sock, sockets):
super(socket_wrapper, self).__init__()
self.sock = sock
self.fileno = sock.fileno
self.sockets_list = sockets
self.channel = stackless.channel()
self.writable = False
self.error = False
def remove(self):
self.sock.close()
self.sockets_list.remove(self)
def send(self, data):
self.sock.send(data)
def push(self, bytes):
self.channel.send(self.sock.recv(bytes))
def stackless_accept(accept, handler, recv_size=1024, timeout=0):
sockets = [accept]
while True:
read, write, error = select.select(sockets, sockets, sockets, timeout)
for sock in read:
if sock is accept:
# Accept socket and create wrapper
sock = socket_wrapper(sock.accept()[0], sockets)
# Create tasklett for this connection
tasklet = stackless.tasklet(handler)
tasklet.setup(sock)
# Store socket
sockets.append(sock)
else:
# Send data to handler
sock.push(recv_size)
# Tag all writable sockets
for sock in write:
if sock is not accept:
sock.writable = True
# Tag all faulty sockets
for sock in error:
if sock is not accept:
sock.error = True
else:
pass # should do something here if the main socket is faulty
timeout = 0 if socket else 1
stackless.schedule()
def simple_handler(tsock):
data = ""
while data[-4:] != "\r\n\r\n":
data += tsock.channel.receive()
while not tsock.writable and not tsock.error:
stackless.schedule()
if not tsock.error:
tsock.send("HTTP/1.1 200 OK\r\nContent-length: 8\r\n\r\nHi there")
tsock.remove()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(("192.168.0.192", 8000))
sock.listen(5)
stackless.tasklet(stackless_accept)(sock, simple_handler)
stackless.run()
So i'm toying around with stackless python, writing a very simple webserver to teach myself programming with microthreads/tasklets. But now to my problem, when I run something like ab -n 100000 -c 50 http://192.168.0.192/
(100k requests, 50 concurrency) in apache bench I get something like 6k req/s, the second time I run it I get 5.5k, third time 5k, fourth time, 4.5k, etc. all the way down to 100req/s or something.
The problem goes away when I restart the python script, though.
Now my question is why? Am i forgetting to delete tasklets? I've checked the stackless.getruncount() (and it always seems to return 1, for some reason) so it doesn't seem like there would be any dead tasklets hanging around? I've tried calling .kill() on all tasklets that are done, didn't help. I just can't figure this one out.
import socket
import select
import stackless
import time
class socket_wrapper(object):
def __init__(self, sock, sockets):
super(socket_wrapper, self).__init__()
self.sock = sock
self.fileno = sock.fileno
self.sockets_list = sockets
self.channel = stackless.channel()
self.writable = False
self.error = False
def remove(self):
self.sock.close()
self.sockets_list.remove(self)
def send(self, data):
self.sock.send(data)
def push(self, bytes):
self.channel.send(self.sock.recv(bytes))
def stackless_accept(accept, handler, recv_size=1024, timeout=0):
sockets = [accept]
while True:
read, write, error = select.select(sockets, sockets, sockets, timeout)
for sock in read:
if sock is accept:
# Accept socket and create wrapper
sock = socket_wrapper(sock.accept()[0], sockets)
# Create tasklett for this connection
tasklet = stackless.tasklet(handler)
tasklet.setup(sock)
# Store socket
sockets.append(sock)
else:
# Send data to handler
sock.push(recv_size)
# Tag all writable sockets
for sock in write:
if sock is not accept:
sock.writable = True
# Tag all faulty sockets
for sock in error:
if sock is not accept:
sock.error = True
else:
pass # should do something here if the main socket is faulty
timeout = 0 if socket else 1
stackless.schedule()
def simple_handler(tsock):
data = ""
while data[-4:] != "\r\n\r\n":
data += tsock.channel.receive()
while not tsock.writable and not tsock.error:
stackless.schedule()
if not tsock.error:
tsock.send("HTTP/1.1 200 OK\r\nContent-length: 8\r\n\r\nHi there")
tsock.remove()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(("192.168.0.192", 8000))
sock.listen(5)
stackless.tasklet(stackless_accept)(sock, simple_handler)
stackless.run()
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
两件事情。
首先,请使类名称以大写字母开头。 它更传统,更容易阅读。
更重要的是,在 stackless_accept 函数中,您积累了一个名为
sockets
的Sock
对象的列表
。 这个列表似乎在无限增长。 是的,您有一个remove
,但它并不总是被调用。 如果套接字出现错误,那么它似乎将永远留在集合中。Two things.
First, please make Class Name start with an Upper Case Letter. It's more conventional and easier to read.
More importantly, in the
stackless_accept
function you accumulate alist
ofSock
objects, namedsockets
. This list appears to grow endlessly. Yes, you have aremove
, but it isn't always invoked. If the socket gets an error, then it appears that it will be left in the collection forever.