Cassandra是否使用Heap内存来存储blooms过滤器,100GB数据消耗多少空间?
我了解到 cassandra 使用 blooms 过滤器来提高性能,并将这些过滤器数据存储到物理内存中。
1)cassandra 在哪里存储这个过滤器?(在堆内存中?)
2)这些过滤器消耗多少内存?
I come to know that cassandra uses blooms filter for performance ,and it stores these filter data into physical-memory.
1)Where does cassandra stores this filters?(in heap memory ?)
2)How much memory do these filters consumes?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
运行时,布隆过滤器必须保存在内存中,因为它们的全部目的是避免磁盘 IO。
但是,每个过滤器都与构成每个 SSTable 的其他文件一起保存到磁盘 - 请参阅 http://wiki。 apache.org/cassandra/ArchitectureSSTable
过滤器通常只占数据大小的很小一部分,尽管实际比率似乎变化很大。在我手边的测试节点上,我能找到的最大过滤器是 3.3MB,相当于 1GB 的数据。然而,对于另一个 1.3GB 的数据文件,过滤器只有 93KB...
如果您正在运行 Cassandra,您可以通过在数据目录中查找名为 *-Filter.db 的文件来自行检查过滤器的大小
When running, the Bloom filters must be held in memory, since their whole purpose is to avoid disk IO.
However, each filter is saved to disk with the other files that make up each SSTable - see http://wiki.apache.org/cassandra/ArchitectureSSTable
The filters are typically a very small fraction of the data size, though the actual ratio seems to vary quite a bit. On the test node I have handy here, the biggest filter I can find is 3.3MB, which is for 1GB of data. For another 1.3GB data file, however, the filter is just 93KB...
If you are running Cassandra, you can check the size of your filters yourself by looking in the data directory for files named *-Filter.db