Stackless python网络性能随着时间的推移而下降?

发布于 2024-07-13 16:24:52 字数 2749 浏览 4 评论 0原文

因此,我正在尝试使用 stackless python,编写一个非常简单的网络服务器来教自己使用微线程/tasklet 进行编程。 但现在我的问题是,当我在 apache bench 中运行类似 ab -n 100000 -c 50 http://192.168.0.192/ (100k 请求,50 并发)时,我得到类似 6k req/ 的结果s,我第二次运行它时得到 5.5k,第三次得到 5k,第四次得到 4.5k,等等,一直下降到 100req/s 之类的。

不过,当我重新启动 python 脚本时,问题就消失了。

现在我的问题是为什么? 我忘记删除tasklet了吗? 我已经检查了 stackless.getrunco​​unt() (由于某种原因,它似乎总是返回 1),所以看起来不会有任何死亡的 tasklet 挂在周围? 我尝试在所有已完成的小任务上调用 .kill() ,但没有帮助。 我就是想不通这个问题。

import socket
import select
import stackless
import time

class socket_wrapper(object):
    def __init__(self, sock, sockets):
        super(socket_wrapper, self).__init__()
        self.sock = sock
        self.fileno = sock.fileno
        self.sockets_list = sockets
        self.channel = stackless.channel()
        self.writable = False
        self.error = False

    def remove(self):
        self.sock.close()
        self.sockets_list.remove(self)

    def send(self, data):
        self.sock.send(data)

    def push(self, bytes):
        self.channel.send(self.sock.recv(bytes))

def stackless_accept(accept, handler, recv_size=1024, timeout=0):
    sockets = [accept]

    while True:
        read, write, error = select.select(sockets, sockets, sockets, timeout)

        for sock in read:
            if sock is accept:
                # Accept socket and create wrapper
                sock = socket_wrapper(sock.accept()[0], sockets)

                # Create tasklett for this connection
                tasklet = stackless.tasklet(handler)
                tasklet.setup(sock)

                # Store socket
                sockets.append(sock)

            else:
                # Send data to handler
                sock.push(recv_size)

        # Tag all writable sockets
        for sock in write:
            if sock is not accept:
                sock.writable = True

        # Tag all faulty sockets
        for sock in error:
            if sock is not accept:
                sock.error = True
            else:
                pass # should do something here if the main socket is faulty

        timeout = 0 if socket else 1
        stackless.schedule() 

def simple_handler(tsock):
    data = ""

    while data[-4:] != "\r\n\r\n":
        data += tsock.channel.receive()

    while not tsock.writable and not tsock.error:
        stackless.schedule()

    if not tsock.error:
        tsock.send("HTTP/1.1 200 OK\r\nContent-length: 8\r\n\r\nHi there")
        tsock.remove()

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(("192.168.0.192", 8000))
sock.listen(5)

stackless.tasklet(stackless_accept)(sock, simple_handler)
stackless.run()

So i'm toying around with stackless python, writing a very simple webserver to teach myself programming with microthreads/tasklets. But now to my problem, when I run something like ab -n 100000 -c 50 http://192.168.0.192/ (100k requests, 50 concurrency) in apache bench I get something like 6k req/s, the second time I run it I get 5.5k, third time 5k, fourth time, 4.5k, etc. all the way down to 100req/s or something.

The problem goes away when I restart the python script, though.

Now my question is why? Am i forgetting to delete tasklets? I've checked the stackless.getruncount() (and it always seems to return 1, for some reason) so it doesn't seem like there would be any dead tasklets hanging around? I've tried calling .kill() on all tasklets that are done, didn't help. I just can't figure this one out.

import socket
import select
import stackless
import time

class socket_wrapper(object):
    def __init__(self, sock, sockets):
        super(socket_wrapper, self).__init__()
        self.sock = sock
        self.fileno = sock.fileno
        self.sockets_list = sockets
        self.channel = stackless.channel()
        self.writable = False
        self.error = False

    def remove(self):
        self.sock.close()
        self.sockets_list.remove(self)

    def send(self, data):
        self.sock.send(data)

    def push(self, bytes):
        self.channel.send(self.sock.recv(bytes))

def stackless_accept(accept, handler, recv_size=1024, timeout=0):
    sockets = [accept]

    while True:
        read, write, error = select.select(sockets, sockets, sockets, timeout)

        for sock in read:
            if sock is accept:
                # Accept socket and create wrapper
                sock = socket_wrapper(sock.accept()[0], sockets)

                # Create tasklett for this connection
                tasklet = stackless.tasklet(handler)
                tasklet.setup(sock)

                # Store socket
                sockets.append(sock)

            else:
                # Send data to handler
                sock.push(recv_size)

        # Tag all writable sockets
        for sock in write:
            if sock is not accept:
                sock.writable = True

        # Tag all faulty sockets
        for sock in error:
            if sock is not accept:
                sock.error = True
            else:
                pass # should do something here if the main socket is faulty

        timeout = 0 if socket else 1
        stackless.schedule() 

def simple_handler(tsock):
    data = ""

    while data[-4:] != "\r\n\r\n":
        data += tsock.channel.receive()

    while not tsock.writable and not tsock.error:
        stackless.schedule()

    if not tsock.error:
        tsock.send("HTTP/1.1 200 OK\r\nContent-length: 8\r\n\r\nHi there")
        tsock.remove()

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(("192.168.0.192", 8000))
sock.listen(5)

stackless.tasklet(stackless_accept)(sock, simple_handler)
stackless.run()

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

酒几许 2024-07-20 16:24:52

两件事情。

首先,请使类名称以大写字母开头。 它更传统,更容易阅读。

更重要的是,在 stackless_accept 函数中,您积累了一个名为 socketsSock 对象的列表。 这个列表似乎在无限增长。 是的,您有一个remove,但它并不总是被调用。 如果套接字出现错误,那么它似乎将永远留在集合中。

Two things.

First, please make Class Name start with an Upper Case Letter. It's more conventional and easier to read.

More importantly, in the stackless_accept function you accumulate a list of Sock objects, named sockets. This list appears to grow endlessly. Yes, you have a remove, but it isn't always invoked. If the socket gets an error, then it appears that it will be left in the collection forever.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文