用于大容量的 rrd 工具替代品
我有兴趣知道是否有任何替代 rrdtool 来记录时间序列数据。 我正在寻找可以扩展以监控大量设备的东西。
根据我在这个主题上读到的内容，当您使用大量数据时，rrdtool 会受到 I/O 限制。 由于我设想将其扩展到需要监控的大量设备，因此我很好奇是否有任何替代方案不会阻塞 I/O。 最好基于 SQL，但不是必须的。
谢谢
I am interested in knowing if there is any alternative to rrdtool for logging time series data. I am looking at something that can scale for a large number of devices to monitor.
From what I read on this subject, rrdtool is I/O bound when you hit it with large amounts of data. Since I envision this to scale to a very large number of devices to monitor, I am curious if there's any alternative that would not choke on I/O. Preferable SQL based, but not necessarily.
Thanks
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
有一些时间序列数据库以高可用性和/或可扩展性为目标。
也许看看
There are some time series databases which have high availability and/or scalability as goals.
Maybe have a look at
如果 I/O 性能是主要问题，那么您需要研究类似 rrdcached 在 RRDTools 的当前版本 (1.4) 中可用。
I/O 开销不是所写入数据的函数，毕竟每个数据源每个值 8 个字节。 I/O 带宽来自于整个扇区（通常为 4k）在写出之前需要读入的事实。 突然要写入 8 个字节，您就已经读/写了 8k 字节。
rrdcached 将所有这些写入合并在一起，因此当更新 RRD 时，有用数据（实际 DS 值）与浪费数据（扇区中的备用字节）的比率会降低。
当所有 RRDTools 检测到 rrdcached 正在运行时（通过环境变量），它们都会自动与 rrdcached 一起工作。 这允许它们在需要时触发刷新，例如从数据生成图表时。
虽然切换到基于 SQL 的解决方案可能有助于考虑支持 SQL 所需的额外 I/O。 考虑到您不倾向于在这种随机访问模式中使用 RRD 数据，数据库对于解决该问题来说有点像大锤。 在坚持使用 RRDTool 的同时，您将可以访问所有能够理解并可以使用文件的工具生态系统，这在您已经熟悉它的情况下尤其有用。
If I/O performance is the main worry then you want to look into something like rrdcached which is available in the current version (1.4) of the RRDTools.
The I/O overhead is not a function of the data being written, after all each value 8 bytes per data source. The I/O bandwidth comes from the fact a whole sector (typically 4k) needs to be read in before being written out. Suddenly to write 8 bytes you have read/written 8k bytes.
The rrdcached coalesces all these write together so when an RRD is updated the ratio of useful data (actual DS values) to wasted data (the spare bytes in the sector) is reduced.
All the RRDTools will automatically work with rrdcached when they detect it running (via an environment variable). This allows them to trigger flushes when needed, for example when generating a graph from the data.
While switching to an SQL based solution may help consider the extra I/O that will be required to support SQL. Considering you don't tend to use RRD data in that sort of random access pattern a database is a bit of a sledgehammer for the problem. While sticking with RRDTool will keep access to all the eco-system of tools that understand and can work with the files, which is useful especially if you are already familiar with it.
我的一个朋友不久前在 SQL 后端做了一些工作来存储循环数据：http://rrs.decibel。 org
但是，我怀疑既然您询问“要监视的设备”，您可能正在寻找更完整的解决方案。
A friend of mine did some work a while ago on a SQL backend to store round robin data: http://rrs.decibel.org
However, I suspect that since you're asking about "devices to monitor", you may be looking for a more complete solution.
如果每秒 I/O 操作是您的主要瓶颈，并且您使用的是 Linux，那么有一个简单的技巧，只消耗您的内存。 使用 tmpfs 挂载来暂存 RRD 写入。
所有 I/O 操作都将在内存中完成，不会产生磁盘 I/O 中发现的任何瓶颈（这甚至比使用固态磁盘更快）。 然后，您可以使用 cron 作业和 rsync 每隔几分钟仅将更改的 RRD 复制到磁盘一次。
创建目录
使用适当的选项创建最大 500MB RAM 文件系统
将旧的 RRD 文件复制到新的挂载点
配置您的 rrd -编写应用程序写入新的挂载点
设置一个 cron 作业，每 2 分钟仅将更改的 RRD 同步到磁盘
不要忘记将保存的 RRD 文件复制到在启动 rrd 写入应用程序之前安装点！ 您可能需要编辑该服务的初始化脚本，以确保文件在启动之前就在那里。 如果在文件没有就位的情况下启动，则会创建新的裸文件，并且一旦读取目录被空 RRD 覆盖，您就会感到非常困惑。
如果在某些时候您需要调整 tmpfs 挂载的大小，您可以即时执行此操作：
If I/O operations per second is your main bottleneck and you're using Linux, there's an easy hack that only costs you memory. Use a tmpfs mount to stage your RRD writes.
All the i/o operations will be done in memory and won't incur any of the bottlenecks found in doing disk i/o (this is even faster than using solid state disks). You can then use a cron job and rsync to copy only changed RRDs to disk once every few minutes.
Create the directories
Create a 500MB-maximum RAM filesystem with appropriate options
Copy the old RRD files into the new mount point
Configure your rrd-writing application to write to the new mount point
Set up a cron job to sync only the changed RRDs to disk once every 2 minutes
Don't forget to copy your saved RRD files into the mount point before you start your rrd-writing application! You may need to edit the init script for that service to make sure the files are there before it starts. If it starts without the files in place, new bare ones will be created and you'll be very confused once the read directory gets overwritten with empty RRDs.
If at some point you need to resize the tmpfs mount, you can do that on the fly: