XML 数据传输的最大大小？
有没有人尝试过在 xml 中传递 5GB 的数据。需要传递大量数据的 Web 服务是否使用 XML over http 来传递数据。
我期待着对系统进行更改以传递数据。我很困惑是否应该以 XML 格式传递 5 GB 的数据，因为我的主内存只有 2GB。
应用程序会崩溃吗？
谢谢
Has anyone ever tried passing 5GB of data in a xml. Do web services which require to pass large amount of data use XML over http to pass data.
I am looking forward to making changes in a system to pass data. I am confused if I should pass data in an XML format for 5 GB of data as my main memory is just 2GB.
Will the application break?
Thanks
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
XML 只是一种标记语言/数据格式，没有任何固有的大小限制。如果需要，您可以创建 1000 GB 的 XML 文件。
如果操作 5 GB XML 文件（或任何其他类型的 5 GB 文件）的东西没有被设计来处理大文件，可能会被破坏。一般来说，如果您只是将大文件上传到网络服务，那么应该没问题，因为几乎所有现代文件上传模块都支持在收到上传文件时将其缓存到磁盘，这样整个文件就不需要留在记忆中。但是，一旦将文档放在服务器上，您在解析文档时可能会遇到一些问题，具体取决于您用来进行解析的库。您可能想了解哪种类型的流式 XML 解析器可用于您的 Web 服务/平台（或者甚至专门针对您的 XML 文档格式编写您自己的解析器，因为这样您就可以做出简化的假设，从而限制内存量任何给定时间都需要）。
我想大多数传递大量数据的 Web 服务都不会使用 XML 作为数据传输格式。带宽昂贵，高延迟或长上传时间可能会导致用户体验不佳。因此，我预计此类服务通常会使用优化的二进制格式。只需在发送 XML 文档之前对其应用 gzip 压缩即可获得合理的近似值。
XML is just a markup language/data format, and does not have any inherent size limits. You can make a 1000 GB XML file if you want.
Things that manipulate a 5 GB XML file (or any other type of 5 GB file) may break if they have not been designed to handle large file sizes. In general, if you are just uploading your large file to a web service you should be okay, because nearly any modern file-upload module is going to support caching the upload to disk as it is received so that the whole file doesn't need to be in memory. You may, however, have some issues with parsing the document once you have it on the server, depending upon what library you use to do the parsing. You may want to look into what sort of streaming XML parsers are available for your web service/platform (or even write your own parser specifically targeted at your XML document format, since then you can make simplifying assumptions that let you limit the amount of memory required at any given time).
I would imagine that most web services that pass large amounts of data around would not use XML as the data transfer format. Bandwidth is expensive, and high latency or long upload times can make for a poor user experience. So I'd expect such services to more typically use an optimized binary format. A reasonable approximation of this could be obtained by simply applying gzip compression to your XML document before you send it.
我对大型 XML 文件有一些经验，但可能不是 5GB。
如果这是使用 XML 的现有系统，那么在从 XML 更改为其他格式之前请仔细考虑，因为更改本身可能会带来更多麻烦而不是其价值。压缩文件将大大有助于网络传输。 gzip 压缩的 XML 文件与专有的二进制格式一样高效。
您可能的瓶颈是文件的解析和处理。如果 XML“记录”彼此独立（例如，如果这是一个很长的 xml 类型列表），那么您应该能够使用流式 XML 解析器来避免将所有内容加载到内存中。还可以考虑使用“非验证”解析器（或关闭验证）来提高性能。
如果您可以在 XSLT 中进行任何文件处理，那么您可能会发现这比将整个文件解析到程序中进行操作更好。
根据网络传输时间，考虑使用可靠的网络传输，例如 FTP 或 BitTorrent。如果 HTTP 失去连接，您可能必须重新开始。
I've had some experience with large XML files, but maybe not 5GB.
If this is an existing system using XML, then think hard before changing from XML to some other format because the change itself might be more trouble than its worth. Compressing the file will go a long way to helping with the network transfer. A gzipped XML file can be just as efficient as a proprietary binary format.
Your likely bottleneck will be the parsing and processing of the file. If the XML "records" are independent of each other (e.g. if this is a long list of xml types) then you should be able to use a streaming XML parser to avoid loading everything into memory. Also consider using a "non-validating" parser (or switching off validation) to improve performance.
If you can do any of the file processing in XSLT, then you might find that works better than parsing the whole file into a program for manipulation.
Depending on the network transfer time, consider using a reliable network transport such as FTP or BitTorrent. If HTTP loses the connection, you might have to start over.
通过 Web 服务传输 5gb 似乎很大，但您可以压缩 XML，这应该会显着减小 XML 的大小。
压缩 XML 指标 。或者
，您可以采用不同的方法来完成它，例如使用 Windows 任务管理器或 zips > 的 Linux contrab 作业进行每晚/每周的计划任务。 FTP 上的文件吗？然后在另一端有另一个运行以导入数据的计划任务。或者可以有一个网页或 Web 服务，用于触发在接收服务器上启动导入
5gb seems like a large amount to transfer over a web service but you can compress the XML which should significantly reduce the size as XML.
Compression XML metrics .
Alternatively, could you do it as a different approach like a nightly / weekly scheduled task using windows task manager or linux contrab job which zips > FTP's the file across? Then on the other end have another schedule task which runs to import the data. or could have a web page or web service which is used to trigger an import to start on the receiving server