如何在Python中构建能够进行热代码交换的Twisted服务器?
我开发了一套音频流服务器,它们都使用Twisted,当然它们都是Python的。它们可以工作,但是一个问题一直困扰着我,当我在运行的服务器中发现一些错误,或者我想向服务器添加一些东西时,我需要停止它们并启动。与 HTTP 服务器不同,可以随时重新启动它们,但对于音频流服务器则不行。一旦我重新启动流媒体服务器,这意味着我的用户将遇到断开连接。
我确实尝试设置一个沙井(Twisted 服务器的 ssh 服务,您可以登录并在控制台中键入 Python 代码来执行某些操作),然后连接到控制台,即时重新加载 Python 模块。有时它会起作用,但很难控制。你永远不知道服务器中有多少旧类的实例,其中一些可能很难访问,并且类的关系会非常复杂。另外,它可能在某些情况下有效,但有时你确实需要重新启动服务器,例如,你正在使用选择器反应器运行服务器,而你想使用 epoll 反应器运行它,那么你必须重新启动它。另一个例子,当内存使用率过高时,你也必须重新启动它们。
为了构建这样的系统,我脑海中浮现出一个想法,我认为可以将这些连接和数据从一个进程移交给另一个进程。例如:
我们有一个名为 Broadcasting 的服务器,运行实例的版本为 rev.123,我们希望将其替换为 rev.124。
Broadcasting rev.123 is running....
Startup Broadcasting rev.124 ....
Broadcasting rev.124 is stand by
Hand over connections from instance of rev.123 to instance of rev.124
Stop Broadcasting rev. 123 instance
这可能吗?我不知道套接字句柄的生命周期是否绑定到进程,我认为当创建者进程被杀死时,进程创建的套接字将被关闭,但我不确定。如果可能的话,是否有任何设计此类热代码交换机制的指南或文章?有什么可以实现我想要的 Twisted 已经完成的事情吗?
谢谢。
I've developed a set of audio streaming server, all of them are using Twisted, and they are in Python, of course. They work, but a problem keeps troubling me, when I found some bugs there in the running server, or I want add something into the server, I need to stop them and start. Unlike HTTP servers, it's okay to restart them whenever, but not okay with audio streaming servers. Once I restart my streaming server, it means my users will encounter a disconnection.
I did try to setup a manhole (a ssh service for Twisted servers, you can login and type Python code in the console to do something), and connect to the console, reload Python modules on the fly. It works sometimes, but hard to control. You never know how many instances of old class are there in the server, and some of them might be hard to reach, and relationships of class would be very complex. Also, it may works in some situations, but sometimes you really need to restart server, for example, you are running the server with selector reactor, and you want to run it with epoll reactor instead, then you have to restart it. Another example, when the memory usage goes too high, you have to restart them, too.
To build such system, I have an idea comes in my head, I'm thinking is that possible to hand over those connections and data from a process to another. For example:
We have a Server named Broadcasting, and the running instance is under rev.123, and we want replace it with rev.124.
Broadcasting rev.123 is running....
Startup Broadcasting rev.124 ....
Broadcasting rev.124 is stand by
Hand over connections from instance of rev.123 to instance of rev.124
Stop Broadcasting rev. 123 instance
Is this possible? I have no idea that does lifetime of socket handles bound to processes or not, I thought sockets created by a process will be closed when the creator process is killed, but I'm not sure. If it is possible, are there any guidelines or articles for designing such kind of hot code swapping mechanism? And is there something can achieve what I want for Twisted already be done?
Thanks.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
我在 PyCon 2004 上对此进行了演讲 。 我们还努力向 Twisted 本身添加更多功能来帮助解决此问题。
I gave a talk about this at PyCon 2004. There's also some effort to add more functionality to help with this to Twisted itself.