在 PHP 中处理内存数据
我正在编写一个 PHP 框架，可用于可视化大量数据。我无法将所有数据提取到内存中并对其进行处理（如排序、过滤等）。所以我有一个 sql 查询生成器，它构造 sql 查询并将所有处理部分推送到 sql 服务器。有什么办法可以用 PHP 做到这一点吗？或者是否有某种 C 库可用，它以二进制形式存储数据并减少处理时的内存使用量？
I am writing a PHP framework which can be used to visualize huge amounts of data. I cannot fetch all the data into memory and process it (like sort,filter etc). So I have a sql query builder which constructs sql queries and pushes all the processing part to the sql server. Is there a way I can do it with PHP? Or is there some kind of C library available which stores data in binary and reduces the memory usage while processing?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
我在 PHPExcel 中大量使用缓存引擎（使用 APC、对象序列化、php://temp、磁盘存储、内存 SQLite、memcache 和其他选项），它可以处理大量（数百万）单元格对象。问题是通过尽可能多地考虑 php 的有限内存来减少内存开销，同时尝试保持合理的执行速度：任何在 PHP 内存之外存储数据的单元缓存都将不可避免地增加执行速度开销。
使用 igbinary 序列化数据是一个很好的权衡，但您仍然可能会遇到内存限制。它与 PHP 内置的序列化速度差不多，但在压缩数据方面效率更高。
我想说的是，尽可能多地对数据库进行操作，因为数据库就是为此类工作而设计的。
I make heavy use of a caching engine in PHPExcel (using APC, serialization of objects, php://temp, disk storage, in-memory SQLite, memcache and other options), which can work with huge numbers (several millions) of cell objects. The problem is reducing the memory overhead by factoring as much out of php's limited memory as possible, while trying to keep execution speed reasonable: any cell cache that stores data outside of PHP memory will invariable add execution speed overhead.
Serialization of data using igbinary is a good trade-off, but you can still hit memory limits. It's about as fast as PHP's built in serialize, but a lot more efficient in terms of compressing the data.
I'd say to do as much on the database as you possibly can, because databases are designed for this kind of work.
Zend_Db_Select 是 Zend_Db 和 Zend Framework 的一部分，允许您创建通过推送和修改其中的一部分，以编程方式您的查询（您提到的查询生成器）。
Zend_Db_Select is part of Zend_Db and Zend Framework and allows you to create programatically your query (the query builder that you mention), by pushing and modifying parts of it.