套接字连接可以复用吗?
是否可以复用 sa ocket 连接?
我需要建立与雅虎信使的多个连接,并且我正在寻找一种有效地完成此操作的方法,而不必为每个客户端连接保持打开的套接字。
到目前为止,我必须为每个客户端使用一个套接字,并且这无法扩展到超过 50,000 个连接。
哦,我的解决方案是针对电信公司的,所以我需要至少达到 250,000 到 500,000 个连接,
我计划将多个 IP 地址绑定到单个 NIC,以打破每个 IP 地址的 65k 端口限制。
请我帮忙,我能得到的见解。
**我在这个网站上的大多数其他问题都没有得到解答:) **
谢谢
Is it possible to multiplex sa ocket connection?
I need to establish multiple connections to yahoo messenger and i am looking for a way to do this efficiently without having to hold a socket open for each client connection.
so far i have to use one socket for each client and this does not scale well above 50,000 connections.
oh, my solution is for a TELCO, so i need to at least hit 250,000 to 500,000 connections
i'm planing to bind multiple IP addresses to a single NIC to beat the 65k port restriction per IP address.
Please i would any help, insight i can get.
**most of my other questions on this site have gone un-answered :) **
Thanks
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(5)
这是一个关于在严重情况下扩展的有趣问题。
您本质上是在问:“我如何建立与互联网服务的 N 个连接,其中 N >= 250,000”。
有效且高效地做到这一点的唯一方法是集群。 您无法在单个主机上执行此操作,因此您需要能够将客户群分割并分区到多个不同的服务器中,以便每个服务器仅处理一个子集。
这个想法是让单个服务器保持尽可能少的打开连接(均匀分布连接),同时保持足够的连接,通过将服务器间通信保持在最低水平来使您托管的任何服务都可行。 这意味着任何两个相关的连接(例如两个经常相互通信的帐户)都必须位于同一主机上。
您将需要可以处理此问题的服务器和网络基础设施。 您将需要一个 IP 地址子网,每个服务器都必须与互联网进行无状态通信(即您的路由器不会执行任何 NAT,以免跟踪 250,000 个以上的连接)。
您必须与 AOL 联系。 如果不考虑切断您的连接,AOL 就无法处理这种级别的连接。 任何这种规模的服务都必须与 AOL 协商,以便您和他们都能够处理连接。
您应该研究一些 I/O 复用技术。 我想到了 Kqueue 和 epoll。
为了编写这个大规模并发和电信级解决方案,我建议研究 erlang。 Erlang 是为诸如此类的情况(多服务器、大规模多客户端、大规模多线程电信级软件)而设计的。 它目前用于运行爱立信电话交换机。
This is an interesting question about scaling in a serious situation.
You are essentially asking, "How do I establish N connections to an internet service, where N is >= 250,000".
The only way to do this effectively and efficiently is to cluster. You cannot do this on a single host, so you will need to be able to fragment and partition your client base into a number of different servers, so that each is only handling a subset.
The idea would be for a single server to hold open as few connections as possible (spreading out the connectivity evenly) while holding enough connections to make whatever service you're hosting viable by keeping inter-server communication to a minimum level. This will mean that any two connections that are related (such as two accounts that talk to each other a lot) will have to be on the same host.
You will need servers and network infrastructure that can handle this. You will need a subnet of ip addresses, each server will have to have stateless communication with the internet (i.e. your router will not be doing any NAT in order to not have to track 250,000+ connections).
You will have to talk to AOL. There is no way that AOL will be able to handle this level of connectivity without considering cutting your connection off. Any service of this scale would have to be negotiated with AOL so both you and they would be able to handle the connectivity.
There are i/o multiplexing technologies that you should investigate. Kqueue and epoll come to mind.
In order to write this massively concurrent and teleco grade solution, I would recommend investigating erlang. Erlang is designed for situations such as these (multi-server, massively-multi-client, massively-multithreaded telecommunications grade software). It is currently used for running Ericsson telephone exchanges.
虽然您可以在套接字上侦听多个传入连接请求,但建立连接后,它将服务器上的唯一端口连接到客户端上的唯一端口。 为了复用连接,您需要控制管道的两端,并拥有一种允许您将上下文从一个虚拟连接切换到另一个虚拟连接的协议,或者使用不关心客户端身份的无状态协议。 在前一种情况下,您需要在应用程序层中实现它,以便可以重用现有连接。 在后一种情况下,您可以使用代理来跟踪哪个服务器响应发送到哪个客户端。 由于您要连接到 Yahoo Messenger,我认为您无法执行此操作,因为它需要经过身份验证的连接,并且假定每个连接对应于一个用户。
While you can listen on a socket for multiple incoming connection requests, when the connection is established, it connects a unique port on the server to a unique port on the client. In order to multiplex a connection, you need to control both ends of the pipe and have a protocol that allows you to switch contexts from one virtual connection to another or use a stateless protocol that doesn't care about the client's identity. In the former case you'd need to implement it in the application layer so that you could reuse existing connections. In the latter case you could get by using a proxy that keeps track of which server response goes to which client. Since you're connecting to Yahoo Messenger, I don't think you'll be able to do this since it requires an authenticated connection and it assumes that each connection corresponds to a single user.
如果另一端支持此类操作,则只能通过单个套接字复用多个连接。
换句话说,它是一个函数协议 - 套接字没有任何本地支持。
我怀疑雅虎通讯协议是否支持它。
另一种方法(单个 NIC 上的多个 IP)是设计您自己的多路复用协议,并拥有从多路复用协议转换为 yahoo 协议的卫星服务器。
You can only multiplex multiple connections over a single socket if the other end supports such an operation.
In other words it's a function protocol - sockets don't have any native support for it.
I doubt yahoo messenger protocol has any support for it.
An alternative (to multiple IPs on a single NIC) is to design your own multiplexing protocol and have satellite servers that convert from the multiplex protocol to the yahoo protocol.
我将采用您可以考虑的另一种方法(取决于您的绝望程度)。
请注意,操作系统 TCP/IP 实现需要是通用的,但您只对非常具体的用例感兴趣。 因此,在应用程序代码中实现 TCP/IP 的简化版本(它只处理您的用例,但做得很好)可能是有意义的。
例如,如果您使用的是 Linux,则可以将几个 IP 地址路由到 tun 接口,并让应用程序处理该 tun 接口的 IP 数据包。 这样,您就可以在应用程序中完全实现 TCP/IP(针对您的用例进行了优化),并避免任何操作系统对打开连接数量的限制。
当然,自己完成 TCP/IP 需要做相当多的工作,但这实际上取决于您的绝望程度 - 即您可以投入多少硬件来解决这个问题。
I'll trow in another approach you could consider (depending on how desperate you are).
Note that operating system TCP/IP implementations need to be general purpose, but you are only interested in a very specific use-case. So it might make sense to implement a cut-down version of TCP/IP (which only handles your use-case, but does that very well) in your application code.
For example, if you are using Linux, you could route a couple of IP addresses to a tun interface and have your application handle the IP packets for that tun interface. That way you can implement TCP/IP (optimised for your use-case) entirely in your application and avoid any operating system restriction on the number of open connections.
Of course, it's quite a bit of work doing the TCP/IP yourself, but it really depends on how desperate you are - i.e. how much hardware can you afford to throw at the problem.
500,000 个任意 yahoo Messenger 连接 - 您的电信公司是否代表 Yahoo 这样做? 似乎无论多年来已经存在的解决方案都应该在摩尔定律的帮助下进行扩展 - 据我所知,所有 IM 客户端在很长一段时间内都非常有效,而且需求并没有紧迫的增长我能想到。
为什么这不是一个可以通过硬件加传统解决方案来解决的合理问题呢?
500,000 arbitrary yahoo messenger connections - is your telco doing this on behalf of Yahoo? It seems like whatever solution has been in place for many years now should be scalable with the help of Moore's Law - and as far as I know all the IM clients have been pretty effective for a long time, and there's no pressing increase in demand that I can think of.
Why isn't this a reasonable problem to address with hardware plus traditional solutions?