Tornado 真的是非阻塞的吗?

发布于 2024-09-17 19:21:40 字数 486 浏览 13 评论 0原文

Tornado 宣传自己是“一个相对简单的、非阻塞 Web 服务器框架”,旨在解决 C10k 问题。然而,查看他们包装 MySQLdb 的数据库包装器,我遇到了以下代码:

def _execute(self, cursor, query, parameters):
    try:
        return cursor.execute(query, parameters)
    except OperationalError:
        logging.error("Error connecting to MySQL on %s", self.host)
        self.close()
        raise

据我所知,对 MySQLdb 的调用(构建于 libmysqlclient 之上)是阻塞。

我是否正确地认为长时间运行的查询会使整个 Tornado 服务器在完成之前没有响应,或者代码有魔法吗?

Tornado advertises itself as "a relatively simple, non-blocking web server framework" and was designed to solve the C10k problem. However, looking at their database wrapper, which wraps MySQLdb, I came across the following piece of code:

def _execute(self, cursor, query, parameters):
    try:
        return cursor.execute(query, parameters)
    except OperationalError:
        logging.error("Error connecting to MySQL on %s", self.host)
        self.close()
        raise

As far as I know calls to the MySQLdb, which is built on top of libmysqlclient, are blocking.

Am I right in thinking that a long-running query would render the entire Tornado server unresponsive until it finishes or is there magic on the code?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

生生不灭 2024-09-24 19:21:40

如果您在顶部编写非阻塞代码,那么龙卷风就是非阻塞的,例如。使用 asyncmongo@tornado.web.asynchronous 装饰器。 Tornado 作为一个框架为此提供了工具。

原作者之一布雷特·泰勒写道

我们尝试了不同的异步数据库方法,但最后决定
在 FriendFeed 上同步,因为通常如果我们的数据库查询是
积压我们的请求,我们的后端无法扩展到负载
反正。足够慢的事物被抽象分离
我们通过异步 HTTP 异步获取的后端服务
模块。

确实,Tornado 不包含非阻塞数据库层;事实上,与 Django 的 ORM 不同,数据库层根本不是 Tornado 框架的组成部分。是的,Tornado 附带了阻塞 MySQL 包装器,因为 FriendFeed 恰好使用了它,但它更多的是一个外部库,而不是核心功能。我很确定大多数人都在使用其他东西来访问数据库。

Tornado is non-blocking if you write non-blocking code on the top if it, eg. using asyncmongo and @tornado.web.asynchronous decorator. Tornado as a framework provides tools for that.

Bret Taylor, one of the original authors, writes:

We experimented with different async DB approaches, but settled on
synchronous at FriendFeed because generally if our DB queries were
backlogging our requests, our backends couldn't scale to the load
anyway. Things that were slow enough were abstracted to separate
backend services which we fetched asynchronously via the async HTTP
module.

It's true that Tornado doesn't include a non-blocking database layer; in fact the database layer is not integral part of the Tornado framework at all, as opposed to e.g. Django's ORM. Yes, Tornado ships with blocking MySQL wrapper because that's what FriendFeed happened to use, but it's more an external library than core functionality. I'm pretty sure most people are using something else for database access.

别想她 2024-09-24 19:21:40

是的,如果没有其他措施,服务器将等待查询完成执行。这并不意味着 Tornado 不是一个非阻塞的 Web 服务器。

“非阻塞 Web 服务器”不会阻塞网络 I/O(如果它提供静态文件服务,则可能对磁盘 I/O 有一些规定)。这并不意味着您可以在应用程序中立即执行违反因果关系的指令。

进行数据库调用需要时间,就像读取文件、格式​​化字符串、处理模板等需要时间一样。在与服务器主事件循环相同的线程中执行任何这些操作都将阻止循环继续进行,直到完成为止。

Yes, absent other measures, the server will wait for the query to finish executing. That does not mean Tornado is not a non-blocking web server.

A "non-blocking web server" doesn't block on network I/O (and may have some provision for disk I/O if it does static file serving). That does not mean you get instant, causality-violating instruction execution in your application.

Doing a database call takes time, just like reading files, formatting strings, processing templates, etc. takes time. Doing any of these things in the same thread as the server's main event loop will prevent the loop from moving on until you're done.

葬シ愛 2024-09-24 19:21:40

Tornado 是非阻塞的,但仅限于某些 IO 操作,例如读取或写入套接字文件。

如果你希望代码中的所有内容都是非阻塞的,你必须自己设计。但是如果你的代码是计算密集型的,那么非阻塞对你来说毫无意义。这种情况你可能会使用多进程。

请记住一件事,非阻塞只是意味着服务器发送/接收日期不会阻塞。如果你不能让你的代码成为非阻塞的,那么你的整个应用程序就会在某种程度上阻塞。

Tornado is nonblocking,but just limited to some IO operation like read or write to a socket file.

if you want everything in your code nonblocking,you have to design by yourself.but if your code is computation intensively, then nonblocking is meaningless for you. this situation you might use multi-process.

remember one thing nonblocking just mean the server send/receive date won’t blocking. If you can’t make your code nonblocking,then your whole application is someway blocking.

神魇的王 2024-09-24 19:21:40

是的;这根本不是一个完全非阻塞的 Web 服务器。

非阻塞 Web 服务器不会阻塞,使用非阻塞 API 进行文件 I/O、数据库访问等,以确保必须等待某些内容完成的一个请求不会阻止其他请求的处理。这适用于可能阻塞服务器的一切,包括数据库访问。

在进行非阻塞数据库访问时,没有什么比“违反因果关系”更愚蠢的事情了。运行与一个请求相关的非阻塞查询并在该请求仍在运行时处理其他请求是非常有意义的。实际上,这通常意味着与数据库后端建立多个连接。

请注意,如果您尝试运行一万个并发请求,请小心:大多数数据库后端无法应对这一点。如果您有几十个并行运行的数据库请求,您可能需要连接池之类的东西,以允许 Web 服务器建立大量数据库连接,而不会淹没后端。这将导致请求阻塞,在队列中等待获取数据库访问权限,但这样做意味着它不会阻塞整个服务器,而只会阻塞需要数据库的请求。

Yes; this is not a fully non-blocking web server at all.

A non-blocking webserver doesn't block, using non-blocking APIs for file I/O, database access, and so on, to ensure that one request that has to wait for something to finish doesn't prevent other requests from being processed. This applies to everything that might block the server, including database access.

There's nothing as silly as "causality violation" in having non-blocking database access; it makes perfect sense to run a non-blocking query related to one request, and to process other requests while that's still running. In practice, this will usually mean making multiple connections to the database backend.

Note that if you're trying to run ten thousand concurrent requests, be careful: most database backends can't cope with this. If you have more than a few dozen database requests to run in parallel, you probably want something like a connection pooler, to allow the web server to make lots of database connections without swamping the backend. This will cause requests to block, waiting in a queue to get database access, but doing it this way means it's not blocking the whole server--just the requests that need the database.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文