Linux 中的线程/套接字限制
首先:对不起我的英语。
伙计们,我在 POSIX 套接字和/或 pthreads 方面遇到了麻烦。我正在嵌入式设备(ARM9 CPU)上进行开发。在设备上将运行多线程 TCP 服务器,并且它将能够处理大量传入连接。服务器从客户端获取连接并增加计数器变量(unsigned int counter
)。客户端例程将在单独的线程中运行。所有客户端将使用 1 个单例类实例(在此类中将打开和关闭相同的文件)。客户端处理文件,然后客户端线程关闭连接套接字,并调用pthread_exit()
。 所以,我的 TCP 服务器无法处理超过 250 个线程(计数器 = 249 +1(服务器线程)。并且我收到“资源暂时不可用”。有什么问题吗?
First of all: sorry for my English.
Guys, I have a trouble with POSIX sockets and/or pthreads. I'm developing on an embedded device(ARM9 CPU). On the device will work multithread TCP server, and it will be able to process a lot of incoming connections. Server gets connection from client and increase counter variable(unsigned int counter
). Clients routines will run in separate threads. All clients will use 1 singleton class instance(in this class will be opened and closed same files). Clients works with files, then client thread closes connection socket, and calls pthread_exit()
.
So, my TCP server can't handle more than 250 threads(counter = 249 +1(server thread). And I got "Resource temporary unavailable". What's the problem?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
每当您达到线程限制 - 或者如上所述,由于线程数量而耗尽虚拟进程地址空间 - 您就......做错了。更多线程无法扩展。尤其是在进行嵌入式编程时。您可以改为处理线程池上的请求。使用
poll(2)
在更少的线程上处理许多连接。这是一个相当成熟的领域,库(如 ACE、asio)一直在利用这种模型,这是有充分理由的。“每个请求一个线程”模型之所以流行,主要是因为它(被认为的)简单的设计。
不过,只要您在单个逻辑线程(有时称为链)上保持连接,就没有真正的区别。
另外,如果请求的处理不涉及阻塞操作,那么您永远不会比在单个线程上进行轮询和处理更好:您可以使用bind/accept的“积压”功能来让内核 担心您的待处理连接! (注意:这假设是单核 CPU,在双核 CPU 上,这种处理在每个 CPU 一个线程上是最佳的)
编辑添加回复:
如果是这种情况,您确实应该仔细检查是否正确加入或分离所有线程。还要考虑同步对象;如果您始终忘记调用相关的 pthread *_destroy 函数,即使不需要它,您也会遇到限制。这当然会导致资源泄漏。有些工具也许能够帮助您发现它们(我想到的是 vlagrind/helgrind)
Whenever you hit the thread limit - or as mentioned run out of virtual process address space due to the number of threads - you're.... doing it wrong. More threads don't scale. Especially not when doing embedded programming. You can handle requests on a thread pool instead. Use
poll(2)
to handle many connections on fewer threads. This is prettty well-trod territory and libraries (like ACE, asio) have been leveraging this model for good reasonThe 'thread-per-request' model is mainly popular because of it's (perceived) simple design.
As long as you keep connections on a single logical thread (sometimes known as a strand) there is no real difference, though.
Also, if the handling of a request involves no blocking operations, you can never do better than polling and handling on a single thread after all: you can use the 'backlog' feature of bind/accept to let the kernel worry about pending connections for you! (Note: this assumed a single core CPU, on a dual core CPU this kind of processing would be optimal with one thread per CPU)
Edit Addition Re:
If that's the case, you should really double check that you are joining or detaching all threads properly. Also think of the synchronization objects; if you consistently forget to call the relevant pthread *_destroy functions, you'll run into the limits even without needing it. That would of course be a resource leak. Some tools may be able to help you spot them (vlagrind/helgrind come to mind)
使用ulimit -n检查文件系统句柄的数量。如果数量太低,您可以为当前会话增加该数量。
您还可以编辑 /etc/security/limits.conf 并设置永久限制
Use ulimit -n to check the number of file system handles. You can increase it for your current session if the number is too low.
Also you can edit /etc/security/limits.conf and to set a permanent limit
通常,在 32 位系统上遇到的第一个限制是使用默认堆栈大小时虚拟地址空间不足。
尝试在创建线程时显式指定堆栈大小(小于 1 MB)或使用“ulimit -s”设置默认堆栈大小。
另请注意,您需要 pthread_detach 或 pthread_join 线程,以便释放所有资源。
Usually, the first limit you are hitting on 32-bit systems is that you are running out of virtual address space when using default stack sizes.
Try explicitly specifying the stack size when creating threads (to less than 1 MB) or setting the default stack size with "ulimit -s".
Also note that you need to either pthread_detach or pthread_join your threads so that all resources will be freed.