如何在 Node.js/RoR 中监控 20 个网站（Ping 或 HTTP）的正常运行时间
例如，每 5 分钟 ping 20 个网站的列表以便了解该网站是否响应 HTTP 202 的最佳方法是什么？
最简单的想法就是将 20 个 URL 保存在数据库中，然后运行数据库并对每个 URL 执行 ping 操作。然而，当一个人不回答时会发生什么？之后的人会怎样？
另外，有没有更好但更简单的解决方案？恐怕该列表可能会增长到 20000 个网站，但我没有足够的时间在需要 ping 的 5 分钟内对所有网站进行 ping 操作。
基本上，我描述的是 PingDom、UptimeRobot 等的工作原理。
我正在使用 node.js 和 Ruby on Rails 构建这个系统。 我还倾向于使用 MongoDB 来保存所有 ping 和监控结果的历史记录。
建议？
非常感谢！
What's the best way to ping a list of 20 websites every 5 minutes (for example) in order to know if the site responds with HTTP 202 or not?
The no brainer idea is to save the 20 URLS in a database and just run the database and ping each one. However, what happen when one doesn't answers? What happens to the ones after that?
Also, is there better but no-brainer solution for this? I'm afraid the list can grow to 20000 websites and then there's not enough time to ping them all in the 5 minutes I need to be pinging.
Basically, I'm describing how PingDom, UptimeRobot, and the likes work.
I'm building this system using node.js and Ruby on Rails.
I'm also inclined to use MongoDB to save the history of all the pings and monitoring results.
Suggestions?
Thanks a bunch!
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（6）
Github
我真的很喜欢 Node.js，我想解决这个问题，并希望很快在 github 上分享一些代码来实现这一目标。请记住，我现在只有一个非常基本的设置，托管在 https://github.com/alfredwesterveld/freakinping
PING(ICMP)
首先我想知道您是否真的想执行 ping(ICMP) 或者您只是想了解网站是否返回代码 200（正常）并测量所需时间。我相信从上下文来看，您并不是真的想做 ping，而只是想做一个 http 请求并测量时间。我问这个问题是因为（我相信）普通用户无法从 node.js/ruby/python 执行 ping 操作，因为我们需要原始套接字（root 用户）从编程语言执行 ping 操作（ICMP）。例如，我在 python 中发现了这个 ping 脚本（我也相信我看到了一个简单的某处的 ruby 脚本虽然我不是一个真正的 ruby 程序员）但需要 root 访问权限。我不相信 Node.js 还存在 ping 模块。
消息队列
要实现这种规模，您需要使用 消息队列，例如 redis、beanstalkd 或 gearmand。在 PingDom 的规模下，一个工作进程不会削减它，但在你的情况下（我假设）一个工作进程就可以了。我认为（假设）由于 C(node.js) 扩展，redis 将是最快的消息队列，但我应该再次将其与 beanstalkd 进行基准测试，beanstalkd 是另一个流行的消息队列（但尚未有 C 扩展）。
如果你达到了这个规模，你可能需要托管多个盒子（很多工作线程/进程）来处理负载，但你还没有达到那个规模，而且 Node.js 的速度非常快。它甚至可以用一个盒子来处理这种负载，尽管我不确定（你需要做/运行一些基准测试）。
数据存储/Redis
我认为这可以在node.js 中很容易地实现（我真的很喜欢node.js）。我这样做的方法是使用 redis 作为我的数据存储，因为它非常快！
使用 node_redis（使用 hredis(node.js) c 库）。我将使用 sadd 将 URL 添加到 redis。
每 5 分钟运行一次任务
这几乎不需要任何努力就可以实现。我会使用 setInterval(callback, delay, [arg], [...]) 来重复测试服务器的响应时间。使用 smembers 从 redis 获取
callback上的所有 URL。我会使用 rpush 将所有 URL（消息）放入消息队列中。检查响应（时间）
我可能不完全理解这句话，但它就是这样。如果一个人失败了，那就只是失败了。您可以尝试在 5 秒后再次检查响应（时间）或其他内容，看看它是否在线。应该为此设计一个精确的算法。之后的 URL 不应与之前的 URL 有任何关系，除非它们位于同一服务器。我猜你也清楚地想到了一些事情，因为这样你就不应该同时将所有这些 URL ping 到同一台服务器，而是将它们排队或进行其他操作。
处理 URL
从工作进程（目前只需一个就足够了）使用 brpop 命令从 redis 获取消息（URL）。检查 URL（消息）的响应时间并从列表中获取下一个 URL（消息）。我可能会同时执行几个请求以加快流程。
Github
I really like node.js and I would like to tackle this problem and hopefully soon share some code on github to achieve this. Keep in mind that I only have a veryy basic setup right now hosted at https://github.com/alfredwesterveld/freakinping
PING(ICMP)
First I would like to know if you want to really do a ping(ICMP) or if you just want to know if the website returns with code 200(OK) and measure the time it takes. I believe from the context that you don't really want to do a ping, but just an http request and measure the time. I ask this because(I believe) pinging from node.js/ruby/python can't be done from normal user because we need raw sockets(root user) to do the pinging(ICMP) from programming language. I for example found this ping script in python(I also believe I saw a simple ruby script somewhere although I am not a really big ruby programmer) but requires root access. I don't believe there is even yet a ping module out there for node.js.
Message Queue
What you need to achieve this kind of scale is to use a message queue like for example redis, beanstalkd or gearmand. At the scale of PingDom one worker process is not going to cut it, but in your case it(I assume) one worker will do. I think(assume) redis will be the fastest message queue because of the C(node.js) extension but then again I should benchmark it against beanstalkd, which is another popular message queue(but does not yet have a C extension).
If you get at that scale you might have to have host multiple boxes(a lot of worker threads/processes) to handle the load but you aren't at that scale yet and node.js is insane fast. It might even be able to handle that load with even one single box, although I don't know for sure(you need to do/run some benchmarks).
Datastore/Redis
I think this could be achieved pretty easily in node.js(I really like node.js). The way I would do this is use redis as my datastore because it is INSANE FAST!
using node_redis(with hredis(node.js) c library). I would Add the URLs to redis using sadd.
Run tasks every 5 minutes
This could be achieved without barely any effort. I would use the
setInterval(callback, delay, [arg], [...])to repeatedly test response time of servers. Get all URLs oncallbackfrom redis using smembers. I would put all the URLs(messages) on the message queue using rpush.Checking Response (Time)
I might not completely understand this sentence but here it goes. If one fails it just fails. You could try to check response(time) again in 5 seconds or something to see if it is online. A precise algorithm for this should be devised. The ones after that should not have anything to do with previous URLs unless the are to the same server. Also something you clearly think about I guess because then you should not ping all those URLs to the same server at the same time but queue them up or something.
Processing URL
From the worker process(for now just one would be suffice) fetch message(URL) from redis using brpop command. check response time for URL(message) and fetch next URL(message) from the list. I would probably do a couple of request simultaneous to speed up the process.
没有“基本方法”，因为您必须处理很多用例：
等不是“基本”工具，如果您想要类似的东西，您可能需要付费或依赖现有的开源替代方案。我确信这一点，因为我自己构建了一个远程监控应用程序。它称为 Uptime，用 Node.js 和 MongoDB 编写，托管在 GitHub (https://github.com/fzaninotto/uptime) 上。它花了几周的努力才开发出来，所以相信我：这不是一件容易的事。
There is no "basic way", since you must handle a lot of use cases:
Pingdom and the like are not "basic" tools, and if you want something similar you may want to pay for it or rely on an existing open-source alternative. I know it for sure because I built a remote monitoring application myself. It's called Uptime, it's written in Node.js and MongoDB, and it's hosted on GitHub (https://github.com/fzaninotto/uptime). It took several weeks of hard work to develop it, so believe me: it is NOT a no-brainer.
使用 zabbix、nagios、blah blah 等监控工具，它们可以大量测量服务器的各种参数。
如果你想在js中实现它，你可以做一个时间间隔的http请求，然后确定http返回状态代码，并使用xpath或正则表达式来验证某些元素
对于ruby来说是正确的，守护进程并使用线程池（多线程思想）和URI打开以查看http代码和内容，使用xpath来验证内容是否行为正确。
use monitoring tools like zabbix, nagios, blah blah which can metric various parameters of your servers in mass numbers.
if u would like to implement it in js, u can do a time interval-ed http request, then to determine http return status code, and use xpath or regex to validate certain element is correct
for ruby, a daemon process and use a thread pool (multithreading idea) and URI open to view the http code and the content, use xpath to validate if the content is behave correctly.
如果您好奇，我创建了一个名为 Pinger 的应用程序来执行此操作。它基于 Ruby on Rails 和 Resque 构建：
https://github.com/austinthecoder/pinger
If you're curious, I've created an app called Pinger that does this. It's built on Ruby on Rails and Resque:
https://github.com/austinthecoder/pinger
有一些免费的优质服务可以为我们提供非常稳定的网站正常运行时间检查和通知。您可以查看此说明并查看 http://fastjoomlahost.com/how-to-监控网站正常运行时间
There are some free quality services what provide us a very stable website up time check and notification. You can check this instruction and review http://fastjoomlahost.com/how-to-monitor-website-up-time
您还可以使用 node-ping-monitor 包在 Node.js 中执行此操作。
You can also do this in Node.js using the node-ping-monitor package.