HAProxy TCP 会话计数停止于 400
我正在尝试使用 HAProxy 进行 TCP 负载平衡。连接进入单个 IP 上的端口 X，然后 HAProxy 使用“leastconn”平衡方法将这些连接平衡到后端，以保持连接数量均匀。这是在 Ubuntu 10.04 x64 上。
我已将内核配置中的 file-max 设置为 700,000。我已将每个进程的 ulimit 调至大约 400,000。我已将 haproxy 配置中的 maxconn 设置为 200,000。它报告说看到这个 maxconn 很好：
show info
Name: HAProxy
Version: 1.3.22
Release_date: 2009/10/14
Nbproc: 1
Process_num: 1
Pid: 1355
Uptime: 0d 4h38m46s
Uptime_sec: 16726
Memmax_MB: 0
Ulimit-n: 400013
Maxsock: 400013
Maxconn: 200000
Maxpipes: 0
CurrConns: 1113
PipesUsed: 0
PipesFree: 0
Tasks: 1113
Run_queue: 1
node: XXXXX
这个前端负载平衡了 5 个后端系统。然而，当每个后端达到 400 个会话时，它只是简单地停止平衡，并且只是推迟额外的连接。我可以通过“smax”统计数据看到这一点。您会注意到，每个会话的最大会话数为 400，而总的最大会话数为 2000：
show stat
#
pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,
protoa,FRONTEND,,,958,2000,2000,12624,6230219,6445523,0,0,0,,,,,OPEN,,,,,,,,,1,1,0,,,,0,0,0,406,
protoa,XXX1672,0,0,191,400,,3222,1249403,1286659,,0,,221,0,664,0,no check,1,1,0,,,,,,1,1,1,,2559,,2,0,,198,
protoa,XXX1674,0,0,192,400,,3106,1242103,1289247,,0,,178,0,535,0,no check,1,1,0,,,,,,1,1,2,,2572,,2,0,,171,
protoa,XXX1707,0,0,193,400,,3043,1266305,1305311,,0,,164,0,492,0,no check,1,1,0,,,,,,1,1,3,,2551,,2,0,,161,
protoa,XXX1782,0,0,189,400,,3046,1236790,1282690,,0,,204,0,619,0,no check,1,1,0,,,,,,1,1,4,,2429,,2,0,,190,
protoa,XXX1851,0,0,193,400,,3060,1235618,1281616,,0,,189,0,570,0,no check,1,1,0,,,,,,1,1,5,,2490,,2,0,,180,
protoa,BACKEND,0,0,958,2000,2000,12624,6230219,6445523,0,0,,956,0,2880,0,UP,5,5,0,,0,17645,0,,1,1,0,,12601,,1,0,,406,
protob,FRONTEND,,,4,6,2000,28,15204,15726,0,0,0,,,,,OPEN,,,,,,,,,1,2,0,,,,0,0,0,2,
protob,XXX1672,0,0,2,2,,5,2313,2322,,0,,0,0,0,0,no check,1,1,0,,,,,,1,2,1,,5,,2,0,,1,
protob,XXX1674,0,0,0,2,,5,3520,3803,,0,,0,0,0,0,no check,1,1,0,,,,,,1,2,2,,5,,2,0,,1,
protob,XXX1707,0,0,0,2,,8,3303,3214,,0,,0,0,0,0,no check,1,1,0,,,,,,1,2,3,,8,,2,0,,1,
protob,XXX1782,0,0,1,2,,5,3529,3745,,0,,0,0,0,0,no check,1,1,0,,,,,,1,2,4,,5,,2,0,,1,
protob,XXX1851,0,0,1,1,,5,2539,2642,,0,,0,0,0,0,no check,1,1,0,,,,,,1,2,5,,5,,2,0,,1,
protob,BACKEND,0,0,4,6,2000,28,15204,15726,0,0,,0,0,0,0,UP,5,5,0,,0,17645,0,,1,2,0,,28,,1,0,,2,
这个限制从何而来？我真的想将数十万个连接限制到这个 haproxy 实例中。 （机器有网络、CPU和RAM来跟上）
I am trying HAProxy for TCP load balancing. Connections come in to port X on a single IP, and the HAProxy then balances these connections to a back-end using the "leastconn" balancing method to keep the number of connections even. This is on Ubuntu 10.04 x64.
I've turned up file-max in the kernel config to 700,000. I've turned up the per-process ulimit to about 400,000. I've turned up maxconn in the haproxy config to 200,000. It reports seeing this maxconn fine:
show info
Name: HAProxy
Version: 1.3.22
Release_date: 2009/10/14
Nbproc: 1
Process_num: 1
Pid: 1355
Uptime: 0d 4h38m46s
Uptime_sec: 16726
Memmax_MB: 0
Ulimit-n: 400013
Maxsock: 400013
Maxconn: 200000
Maxpipes: 0
CurrConns: 1113
PipesUsed: 0
PipesFree: 0
Tasks: 1113
Run_queue: 1
node: XXXXX
This front-end load-balances across 5 back-end systems. However, when it gets to 400 sessions per back-end, it just plain stops balancing, and is just deferring additional connections. I can see this with the "smax" stat. You will note that the maximum number of sessions on each is 400, and the total maximum number of sessions is 2000:
show stat
#
pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,
protoa,FRONTEND,,,958,2000,2000,12624,6230219,6445523,0,0,0,,,,,OPEN,,,,,,,,,1,1,0,,,,0,0,0,406,
protoa,XXX1672,0,0,191,400,,3222,1249403,1286659,,0,,221,0,664,0,no check,1,1,0,,,,,,1,1,1,,2559,,2,0,,198,
protoa,XXX1674,0,0,192,400,,3106,1242103,1289247,,0,,178,0,535,0,no check,1,1,0,,,,,,1,1,2,,2572,,2,0,,171,
protoa,XXX1707,0,0,193,400,,3043,1266305,1305311,,0,,164,0,492,0,no check,1,1,0,,,,,,1,1,3,,2551,,2,0,,161,
protoa,XXX1782,0,0,189,400,,3046,1236790,1282690,,0,,204,0,619,0,no check,1,1,0,,,,,,1,1,4,,2429,,2,0,,190,
protoa,XXX1851,0,0,193,400,,3060,1235618,1281616,,0,,189,0,570,0,no check,1,1,0,,,,,,1,1,5,,2490,,2,0,,180,
protoa,BACKEND,0,0,958,2000,2000,12624,6230219,6445523,0,0,,956,0,2880,0,UP,5,5,0,,0,17645,0,,1,1,0,,12601,,1,0,,406,
protob,FRONTEND,,,4,6,2000,28,15204,15726,0,0,0,,,,,OPEN,,,,,,,,,1,2,0,,,,0,0,0,2,
protob,XXX1672,0,0,2,2,,5,2313,2322,,0,,0,0,0,0,no check,1,1,0,,,,,,1,2,1,,5,,2,0,,1,
protob,XXX1674,0,0,0,2,,5,3520,3803,,0,,0,0,0,0,no check,1,1,0,,,,,,1,2,2,,5,,2,0,,1,
protob,XXX1707,0,0,0,2,,8,3303,3214,,0,,0,0,0,0,no check,1,1,0,,,,,,1,2,3,,8,,2,0,,1,
protob,XXX1782,0,0,1,2,,5,3529,3745,,0,,0,0,0,0,no check,1,1,0,,,,,,1,2,4,,5,,2,0,,1,
protob,XXX1851,0,0,1,1,,5,2539,2642,,0,,0,0,0,0,no check,1,1,0,,,,,,1,2,5,,5,,2,0,,1,
protob,BACKEND,0,0,4,6,2000,28,15204,15726,0,0,,0,0,0,0,UP,5,5,0,,0,17645,0,,1,2,0,,28,,1,0,,2,
Where is this limitation coming from? I really want to cramp hundreds of thousands of connections into this haproxy instance. (The machine has network, CPU and RAM to keep up)
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
因此，阅读 1.3.x 版本的源代码，我发现：
有两个最大值。
一种是全局最大连接数，在命令行上使用 -n 进行设置，在全局配置中使用 maxconn 进行设置。
另一个是每个代理的最大连接数，在命令行上使用 -N 设置，或在配置中为每个代理配置。
具体来说，除了从命令行之外，您无法配置每个代理的默认最大连接数！
默认是...等等...2000！
因此，在 /etc/haproxy/haproxy.cfg 文件中的每个“listen”部分添加“maxconn 200000”可以解决此问题。
请注意，尽管文档非常好，但它并没有很好地解释这一点。具体来说，通过阅读文档，我认为全局 maxconn 也将应用于每个代理，但事实并非如此。全局 maxconn 对于连接总数强制，但必须显式指定给定代理前端的本地最大连接数。
So, reading the source for version 1.3.x, I find out that:
There are two maxima.
One is the global max # connections, set with -n on the command line and maxconn in global config.
The other is the per-proxy max # connections, set with -N on the command line, or configured per proxy in the config.
Specifically, you cannot configure the default max # connections per proxy other than from the command line!
The default is ... wait for it ... 2000!
So adding a "maxconn 200000" to each of the "listen" sections in my /etc/haproxy/haproxy.cfg file solves this problem.
Note that, despite the documentation being pretty good, it did not do a good job of explaining this. Specifically, from reading the documentation, I thought that the global maxconn would be applied to each proxy as well, but not so. The global maxconn is enforced for the total number of connections, but the local max number of connections for a given proxy front end has to be explicitly specified.