如何确定Tomcat中的最佳线程数？
如何确定 Tomcat 中 maxSpare、minSpare 和 maxThreads、acceptCount 等的最佳数量？是否有现有的最佳实践？
我确实理解这需要基于硬件（例如每个核心），并且只能作为对特定硬件进行进一步性能测试和优化的基础。
How does one determine the best number of maxSpare, minSpare and maxThreads, acceptCount etc in Tomcat? Are there existing best practices?
I do understand this needs to be based on hardware (e.g. per core) and can only be a basis for further performance testing and optimization on specific hardware.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
“多少线程问题”是一个相当大且复杂的问题，不能用简单的经验法则来回答。
考虑拥有多少个核心对于往往消耗大量 CPU 的多线程应用程序（例如数字运算等）非常有用。对于网络应用程序来说，这种情况很少见，它通常不是由 CPU 占用，而是由其他因素占用。
一种常见的限制是您与其他外部系统（尤其是数据库）之间的滞后。每次请求到达时，它可能会多次查询数据库，这意味着通过 JDBC 连接传输一些字节，然后等待这些字节到达数据库（即使是在本地主机上，仍然有一个小的延迟） ，然后等待数据库考虑我们的请求，然后等待数据库处理它（数据库本身将等待磁盘寻找到某个区域）等等......
在这段时间里，线程是空闲的，因此另一个线程可以轻松地使用该 CPU 资源来做一些有用的事情。 40% 到 80% 的时间花在等待数据库响应上是很常见的。
连接的另一端也会发生同样的情况。当您的线程将其输出写入浏览器时，客户端连接的速度可能会使您的线程空闲，等待浏览器确认已收到某个数据包。 （几年前这是一个相当大的问题，最近的内核和 JVM 使用更大的缓冲区来防止线程以这种方式闲置，但是 Web 应用程序服务器前面的反向代理，即使只是一个 httpd，对于避免人们使用它也非常有用互联网连接不良会充当 DDOS 攻击:) )
考虑到这些因素，线程数量通常应该比您拥有的核心数量多得多。即使在简单的双核或四核服务器上，您也应该至少配置几十个线程。
那么，是什么限制了您可以配置的线程数量呢？
首先，每个线程（过去）消耗大量资源。每个线程都有一个堆栈，它会消耗 RAM。此外，每个线程实际上都会在堆上分配东西来完成其工作，再次消耗 RAM，并且线程之间的切换（上下文切换）行为对于 JVM/OS 内核来说相当繁重。
这使得“平滑”运行具有数千个线程的服务器变得困难。
鉴于此图，有多种技术（主要是：尝试、失败、调整、重试）来确定您的应用或多或少需要多少线程：
1）尝试了解您的线程在哪里花费了时间。有许多好的工具，但即使是 jvisualvm profiler 也可以是一个很棒的工具，或者是生成摘要计时统计信息的跟踪方面。它们等待外部事物的时间越多，您就越能在空闲时间产生更多线程来使用 CPU。
2) 确定您的 RAM 使用情况。鉴于 JVM 将使用一定量的内存（最显着的是 permgen 空间，通常高达 100 MB，jvisualvm 会再次告诉您），与您使用的线程数量无关，请尝试使用一个线程运行，然后使用 10 个线程运行，然后使用一百，同时使用 jmeter 或其他工具对应用程序进行压力，并查看堆使用量将如何增长。这可能会造成硬性限制。
3）尝试确定一个目标。每个用户请求都需要一个线程来处理。如果你的平均响应时间是每次“get”200ms（最好不要考虑加载图像、CSS 和其他静态资源），那么每个线程每秒能够提供 4/5 个页面。如果每个用户预计每 3/4 秒“单击”一次（取决于它是浏览器游戏还是包含大量长文本的网站？），那么一个线程将“为 20 个并发用户提供服务”，无论这意味着什么。如果在高峰时段，1 分钟内有 500 个单个用户访问您的网站，那么您需要足够的线程来处理该问题。
4）碰撞试验上限。使用 jmeter，在备用虚拟机上配置具有大量线程的服务器，并查看当超过特定限制时响应时间会如何变得更糟。与硬件相比，底层操作系统的线程实现在这里也很重要，但无论怎样，它都会达到这样一个程度：CPU 花费更多的时间来尝试找出要运行的线程，而不是实际运行它，而且这个数字并不是那么令人难以置信。高的。
5) 考虑线程将如何影响其他组件。每个线程可能会使用一个（或者可能多个）数据库连接，数据库是否能够处理 50/100/500 个并发连接？即使您使用的是 nosql 服务器的分片集群，服务器场是否在这些计算机之间提供足够的带宽？还有什么可以与网络应用服务器在同一台机器上运行？阿纳奇httpd？乌贼？数据库本身？数据库的本地缓存代理，如 mongos 或 memcached？
我见过生产中的系统只有 4 个线程 + 4 个备用线程，因为该服务器所做的工作只是调整图像大小，因此它几乎 100% CPU 密集型，而其他系统配置在或多或少相同的硬件上，具有数百个线程，因为 Web 应用程序对外部系统进行了大量 SOAP 调用，并花费大部分时间等待答案。
一旦您确定了大约。对于你的webapp来说，最小和最大线程是最佳的，那么我通常这样配置：
1）基于RAM、其他外部资源和上下文切换实验的限制，有一个绝对不能达到的最大值。因此，使用 maxThreads 将其限制为该数量的大约一半或 3/4。
2) 如果应用程序相当快（例如，它公开通常在几毫秒内发送响应的 REST Web 服务），那么您可以配置较大的acceptCount，最多达到相同数量的 maxThreads。如果您的 Web 应用程序服务器前面有一个负载均衡器，请设置一个小的acceptCount，负载均衡器最好能看到未接受的请求并切换到另一台服务器，而不是让用户在已经很忙的服务器上等待。
3) 由于启动线程（仍然）被认为是一项繁重的操作，因此请使用 minSpareThreads 在高峰时段到来时准备好一些线程。这又取决于您期望的负载类型。甚至可以合理地设置 minSpareThreads、maxSpareThreads 和 maxThreads，以便始终准备好确切数量的线程，从不回收，并且性能是可预测的。如果您在专用计算机上运行 tomcat，则可以提高 minSpareThreads 和 maxSpareThreads 而不会有占用其他进程的危险，否则将其调低，因为线程是与大多数操作系统上运行的其余进程共享的资源。
the "how many threads problem" is quite a big and complicated issue, and cannot be answered with a simple rule of thumb.
Considering how many cores you have is useful for multi threaded applications that tend to consume a lot of CPU, like number crunching and the like. This is rarely the case for a web-app, which is usually hogged not by CPU but by other factors.
One common limitation is lag between you and other external systems, most notably your DB. Each time a request arrive, it will probably query the database a number of times, which means streaming some bytes over a JDBC connection, then waiting for those bytes to arrive to the database (even is it's on localhost there is still a small lag), then waiting for the DB to consider our request, then wait for the database to process it (the database itself will be waiting for the disk to seek to a certain region) etc...
During all this time, the thread is idle, so another thread could easily use that CPU resources to do something useful. It's quite common to see 40% to 80% of time spent in waiting on DB response.
The same happens also on the other side of the connection. While a thread of yours is writing its output to the browser, the speed of the CLIENT connection may keep your thread idle waiting for the browser to ack that a certain packet has been received. (This was quite an issue some years ago, recent kernels and JVMs use larger buffers to prevent your threads for idling that way, however a reverse proxy in front of you web application server, even simply an httpd, can be really useful to avoid people with bad internet connection to act as DDOS attacks :) )
Considering these factors, the number of threads should be usually much more than the cores you have. Even on a simple dual or quad core server, you should configure a few dozens threads at least.
So, what is limiting the number of threads you can configure?
First of all, each thread (used to) consume a lot of resources. Each thread have a stack, which consumes RAM. Moreover, each Thread will actually allocate stuff on the heap to do its work, consuming again RAM, and the act of switching between threads (context switching) is quite heavy for the JVM/OS kernel.
This makes it hard to run a server with thousands of threads "smoothly".
Given this picture, there are a number of techniques (mostly: try, fail, tune, try again) to determine more or less how many threads you app will need:
1) Try to understand where your threads spend time. There are a number of good tools, but even jvisualvm profiler can be a great tool, or a tracing aspect that produces summary timing stats. The more time they spend waiting for something external, the more you can spawn more threads to use CPU during idle times.
2) Determine your RAM usage. Given that the JVM will use a certain amount of memory (most notably the permgen space, usually up to a hundred megabytes, again jvisualvm will tell) independently of how many threads you use, try running with one thread and then with ten and then with one hundred, while stressing the app with jmeter or whatever, and see how heap usage will grow. That can pose a hard limit.
3) Try to determine a target. Each user request needs a thread to be handled. If your average response time is 200ms per "get" (it would be better not to consider loading of images, CSS and other static resources), then each thread is able to serve 4/5 pages per second. If each user is expected to "click" each 3/4 seconds (depends, is it a browser game or a site with a lot of long texts?), then one thread will "serve 20 concurrent users", whatever it means. If in the peak hour you have 500 single users hitting your site in 1 minute, then you need enough threads to handle that.
4) Crash test the high limit. Use jmeter, configure a server with a lot of threads on a spare virtual machine, and see how response time will get worse when you go over a certain limit. More than hardware, the thread implementation of the underlying OS is important here, but no matter what it will hit a point where the CPU spend more time trying to figure out which thread to run than actually running it, and that numer is not so incredibly high.
5) Consider how threads will impact other components. Each thread will probably use one (or maybe more than one) connection to the database, is the database able to handle 50/100/500 concurrent connections? Even if you are using a sharded cluster of nosql servers, does the server farm offer enough bandwidth between those machines? What else will run on the same machine with the web-app server? Anache httpd? squid? the database itself? a local caching proxy to the database like mongos or memcached?
I've seen systems in production with only 4 threads + 4 spare threads, cause the work done by that server was merely to resize images, so it was nearly 100% CPU intensive, and others configured on more or less the same hardware with a couple of hundreds threads, cause the webapp was doing a lot of SOAP calls to external systems and spending most of its time waiting for answers.
Oce you've determined the approx. minimum and maximum threads optimal for you webapp, then I usually configure it this way :
1) Based on the constraints on RAM, other external resources and experiments on context switching, there is an absolute maximum which must not be reached. So, use maxThreads to limit it to about half or 3/4 of that number.
2) If the application is reasonably fast (for example, it exposes REST web services that usually send a response is a few milliseconds), then you can configure a large acceptCount, up to the same number of maxThreads. If you have a load balancer in front of your web application server, set a small acceptCount, it's better for the load balancer to see unaccepted requests and switch to another server than putting users on hold on an already busy one.
3) Since starting a thread is (still) considered a heavy operation, use minSpareThreads to have a few threads ready when peak hours arrive. This again depends on the kind of load you are expecting. It's even reasonable to have minSpareThreads, maxSpareThreads and maxThreads setup so that an exact number of threads is always ready, never reclaimed, and performances are predictable. If you are running tomcat on a dedicated machine, you can raise minSpareThreads and maxSpareThreads without any danger of hogging other processes, otherwise tune them down cause threads are resources shared with the rest of the processes running on most OS.