使用 epoll 的 TCP 服务器的线程和扩展模型
我已经阅读了 C10K 文档以及许多有关扩展套接字服务器的相关论文。所有的道路都指向以下几点：
避免“每个连接一个线程”的经典错误。
优先选择 epoll 而不是 select。
同样，unix 中的传统异步 io 机制可能很难使用。
我的简单 TCP 服务器仅在专用端口上的侦听套接字上侦听客户端连接。收到新连接后，解析请求并发回响应。然后优雅地关闭套接字。
我想我已经很好地掌握了如何使用 epoll 在单线程上扩展它。只需一个循环，为侦听套接字以及现有的客户端连接调用 epoll_wait。返回后，代码将处理新创建的新客户端连接，并根据刚刚收到信号的套接字管理现有连接的状态。也许还有一些逻辑来管理连接超时、套接字的优雅关闭以及每个连接的有效资源分配。看起来很简单。
但是，如果我想扩展它以利用多个线程和多个 CPU 核心，该怎么办？ 浮现在脑海中的核心思想是：
一个专用线程用于侦听 TCP 监听套接字上的传入连接。然后一组 N 个线程（或线程池）来处理所有活动的并发客户端连接。然后发明一些线程安全的方法，其中侦听线程将新连接（套接字）“分派”到可用工作线程之一。 （Windows 中的 IOCP）。工作线程将在它正在处理的所有连接上使用 epoll 循环来执行单线程方法将执行的操作。
我走在正确的轨道上吗？或者是否有一个标准的设计模式可以在多个线程上使用 epoll 来实现 TCP 服务器？
关于监听线程如何将新连接分派到线程池的建议？
I've read the C10K doc as well as many related papers on scaling up a socket server. All roads point to the following:
Avoid the classic mistake of "thread per connection".
Prefer epoll over select.
Likewise, legacy async io mechanism in unix may be hard to use.
My simple TCP server just listens for client connections on a listen socket on a dedicated port. Upon receiving a new connection, parses the request, and sends a response back. Then gracefully closes the socket.
I think I have a good handle on how to scale this up on a single thread using epoll. Just one loop that calls epoll_wait for the listen socket as well as for the existing client connections. Upon return, the code will handle new creating new client connections as well as managing state of existing connections depending on which socket just got signaled. And perhaps some logic to manage connection timeouts, graceful closing of sockets, and efficient resource allocation for each connection. Seems straightforward enough.
But what if I want to scale this to take advantage of multiple threads and multiple cpu cores? The core idea that springs to mind is this:
One dedicated thread for listening for incoming connections on the TCP listen socket. Then a set of N threads (or thread pool) to handle all the active concurrent client connections. Then invent some thread safe way in which the listen thread will "dispatch" the new connection (socket) to one of the available worker threads. (ala IOCP in Windows). The worker thread will use an epoll loop on all the connections it is handling to do what the single threaded approach would do.
Am I on the right track? Or is there a standard design pattern for doing a TCP server with epoll on multiple threads?
Suggestions on how the listen thread would dispatch a new connection to the thread pool?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
epoll操作需要系统调用，它们的成本可能会也可能不会比您自己管理几个fd_set的成本贵。poll也是如此。在计数较低的情况下，每次迭代在用户空间中进行处理会更便宜。第一个问题是，你需要这个吗？如果您通过生成线程来处理每个传入请求来轻松应对现有流量，那么请继续这样做。代码会更简单，并且所有库都会很好地运行。
正如我上面提到的，处理并发请求可能很复杂。如果您想在单个循环中执行此操作，则还需要在生成响应时保证 CPU 饥饿。
如果生成响应的成本很高，那么您提出的调度模型是典型的第一步解决方案。您可以分叉或使用线程。分叉或生成线程的成本不应成为选择池机制的考虑因素：相反，您应该使用这样的机制来限制或排序系统上的负载。
将套接字批处理到多个 epoll 循环上是过度的。如果您如此绝望，请使用多个进程。请注意，可以从多个线程和进程在套接字上
接受。epollmanipulation requires system calls, and their cost may or may not be more expensive than the cost of managing a fewfd_sets yourself. The same goes forpoll. At low counts its cheaper to be doing the processing in user space each iteration.The first question is, do you need this? If you're handily coping with the existing traffic by spawning off threads to handle each incoming request, then keep doing it this way. The code will be simpler for it, and all your libraries will play nicely.
As I mentioned above, juggling simultaneous requests can be complex. If you want to do this in a single loop, you'll also need to make guarantees about CPU starvation when generating your responses.
The dispatch model you proposed is the typical first step solution if your responses are expensive to generate. You can either fork or use threads. The cost of forking or generating a thread should not be a consideration in selecting a pooling mechanism: rather you should use such a mechanism to limit or order the load placed on the system.
Batching sockets onto multiple epoll loops is excessive. Use multiple processes if you're this desperate. Note that it's possible to
accepton a socket from multiple threads and processes.我猜你是在正确的轨道上。但我也认为细节取决于特定情况（带宽、请求模式、单独的请求处理等）。我认为你应该尝试并仔细进行基准测试。
I would guess you are on the right track. But I also think details depend upon the particular situation (bandwidh, request patterns, indifidual request processing, etc.). I think you should try, and benchmark carefully.