在 TCP 负载之间分割 CRLF
我目前正在编写一个低级 HTTP 解析器，并遇到了以下问题：
我正在逐包接收 HTTP 数据，即一次一个 TCP 有效负载。解析此数据时，我使用 HTTP 协议标准搜索 CRLF 来描绘标题行、块数据（在分块编码的情况下）以及双 CRLF 来描绘标题与正文。
我的问题是：我是否需要担心 CRLF 被分割在两个 TCP 数据包有效负载之间的可能性？例如，HTTP 标头将以 CRLFCRLF 结束。有没有可能后续的两个TCP数据包都会有CR，然后是LFCRLF？
我假设是的；这是一个值得担心的情况，因为应用程序 (HTTP) 和 TCP 层彼此相当独立。
任何对此的见解将不胜感激，谢谢！
I'm currently writing a low-level HTTP parser and have run into the following issue:
I am receiving HTTP data on a packet-by-packet basis, i.e. TCP payloads one at a time. When parsing this data, I am using the HTTP protocol standards of searching for CRLF to delineate header lines, chunk data (in the case of chunked-encoding), and the dual CRLF to delineate header from body.
My question is: do I need to worry about the possibility of CRLF being split between two TCP packet payloads? For example, the HTTP header will finish with CRLFCRLF. Is it possible that two subsequent TCP packets will have CR, and then LFCRLF?
I am assuming that yes; this is a case to worry about, since the application (HTTP) and TCP layers are rather independent of each other.
Any insight into this would be highly appreciated, thank you!
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
是的，CRLF 可能会被分割成不同的 TCP 数据包。试想一下单个 HTTP 标头比 TCP MTU 长一个字节的可能性。在这种情况下，只有 CR 的空间，而没有 NL 的空间。
因此，无论您的代码变得多么棘手，它都必须能够处理这种拆分情况。
Yes, it is possible that the CRLF gets split into different TCP packets. Just think about the possibility that a single HTTP header is exactly one byte longer than the TCP MTU. In that case, there is only room for the CR, but not for the NL.
So no matter how tricky your code will get, it must be able to handle this case of splitting.
您使用什么语言工作？它是否没有某种形式的套接字缓冲读取功能，因此您不会遇到此问题？
对你的问题的简短回答是肯定的，理论上你确实需要担心它，因为数据包可能会像这样到达。这是不太可能的，因为大多数 HTTP 端点倾向于在一个数据包中发送标头，并在后续数据包中发送正文。这不太符合惯例，更多的是大多数基于套接字的程序/语言工作方式的本质。
需要记住的一件事是，虽然协议标准对于 CRLF 分离非常明确，但许多实现 HTTP 的人（特别是客户端，但在某种程度上也包括服务器）不知道/关心他们在做什么，并且会不遵守规则。他们倾向于仅用 LF 来分隔行 - 特别是头部和主体之间的空行，我在这个问题中看到的代码段的数量我无法快速计数。虽然这在技术上是违反协议的，但大多数服务器/客户端都会接受这种行为并解决它，因此您也需要这样做。
如果您无法执行某种缓冲读取功能，那么有一些好消息。您所需要做的就是一次将一个数据包读入内存，并将数据标记到前一个数据包上。每次读取数据包时，扫描数据中的双 CRLF 序列，如果没有找到，则读取下一个数据包，依此类推，直到找到头部末尾。这将是相对较小的内存使用量，因为任何请求的头部不应超过 5-6KB，这给定以太网 MTU（平均约为）1450 字节意味着您不需要加载超过 4 或5个数据包放入内存来应对。
What language are you working in? Does it not have some form of buffered read functionality for the socket, so you don't have this issue?
The short answer to your question is yes, theoretically you do have to worry about it, because it is possible the packets would arrive like that. It is very unlikely, because most HTTP endpoints will tend to send the header in one packet and the body in subsequent packets. This is less by convention and more by the nature of the way most socket-based programs/languages work.
One thing to bear in mind is that while the protocol standards are quite clear about the CRLF separation, many people who implement HTTP (clients in particular, but to some degree servers as well) don't know/care what they are doing and will not obey the rules. They will tend to separate lines with LF only - particularly the blank line between the head and the body, the number of code segments I have seen with this problem I could not count up to quickly. While this is technically a protocol violation, most servers/clients will accept this behaviour and work around it, so you will need to as well.
If you can't do some kind of buffered read functionality, there is some good news. All you need to do is read a packet at a time into memory and tag the data on to the previous packet(s). Every time you have read a packet, scan your data for a double CRLF sequence, if you don't find it, read the next packet, and so on until you find the end of the head. This will be relatively small memory usage, because the head of any request shouldn't ever be more than 5-6KB, which given an ethernet MTU of (averaging around) 1450 bytes means you shouldn't ever need to load more than 4 or 5 packets into memory to cope with it.