为高负载网络服务器创建实时统计数据的好方法是什么？
假设我有一堆网络服务器，每个服务器每秒处理 100 个请求，我想查看实时统计数据，例如：
- 过去 5 秒、60 秒、5 分钟等的请求率
- 每个时间窗口再次看到的唯一用户数
或者一般情况下一堆带有时间戳的事件，我想查看实时派生的统计数据 - 最好的方法是什么？
我考虑过让每个 GET 请求在某处更新一个全局计数器，然后以不同的时间间隔对其进行采样，但在事件发生率下，我发现很难获得足够快的分布式计数器。
欢迎任何想法！
添加：服务器是运行 Apache/mod_wsgi 的 Linux，带有 Python (Django) 堆栈。 补充：为了了解我想要跟踪统计数据的事件发生率，它们的发生率超过 10K 个事件/秒。即使以这样的速率增加分布式计数器也是一个挑战。
Say I have a bunch of webservers each serving 100's of requests/s, and I want to see real time stats like:
- Request rate over last 5s, 60s, 5 min etc
- Number of unique users seen again per time window
Or in general for a bunch of timestamped events, I want to see real-time derived statistics - what's the best way to go about it?
I've considered having each GET request update a global counter somewhere, then sampling that at various intervals, but at the event rates I'm seeing it's hard to get a distributed counter that's fast enough.
Any ideas welcome!
Added: Servers are Linux running Apache/mod_wsgi, with a Python (Django) stack.
Added: To give a sense of the event rates I want to track stats for, they're coming in at over 10K events/s. Even incrementing a distributed counter at that rate is a challenge.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
您可能想帮助我们尝试用于 Python Web 应用程序中的应用程序性能监控的代理测试版。
http://newrelic.com
它更深入地研究应用程序性能而不仅仅是 Web 服务器，但由于任何瓶颈都不是它将成为 Web 服务器，但您的应用程序无论如何都会更有用。
免责声明。我在 New Relic 工作，这就是我正在从事的项目。它是一款付费产品，但测试版意味着它的所有功能目前都是免费的。稍后当这种情况发生变化时，如果您不想付费，它们仍然是免费的 Lite 订阅级别，并且为您提供基本的网络指标报告，该报告仍然涵盖您所追求的一些内容。不管怎样，现在是一个很好的机会，可以利用它来调试你的性能。
You might like to help us try out the beta of our agent for application performance monitoring in Python web applications.
http://newrelic.com
It delves more into the application performance rather than just the web server, but since any bottlenecks aren't generate going to be the web server, but your application then that is going to be more useful anyway.
Disclaimer. I work for New Relic and this is the project I am working on. It is a paid product, but the beta means it is free for now with all features. Later when that changes, if you didn't want to pay for it, their is still a Lite subscription level which is free and which gives you basic web metrics reporting which still covers some of what you are after. Anyway, right now would be a great opportunity to make use of it to debug your performance while you can.
几乎所有好的服务器都提供这种开箱即用的功能。例如，Apache 有 mod_status 模块，Glassfish 支持 JMX。此外，有很多用于监控集群的商业软件包，例如Hyperic和Zenoss。
您使用什么网络或应用程序服务器？如果没有这些信息，就很难提供解决方案。
Virtually all good servers provide this kind of functionality out of the box. For example, Apache has the mod_status module and Glassfish supports JMX. Furthermore, there are many commercial packages for monitoring clusters, such as Hyperic and Zenoss.
What web or application server are you using? It is difficult to provide a solution without that information.
看看使用 WebSockets，它们的开销比 HTTP 请求小得多，它们非常适合实时 Web 应用程序。请参阅：http://nodeknockout.com/ 了解基于 Node 的 websocket 示例。
http://en.wikipedia.org/wiki/WebSocket
您需要运行一个守护进程如果你想在你的 apache 服务器上运行它。
另请看一下：
http://kaazing.com/ 如果您不想减少麻烦，但愿意花一些现金。
Look at using WebSockets, their overhead is much smaller than a HTTP request, they are very well suited to real-time web applications. See: http://nodeknockout.com/ for Node based websocket examples.
http://en.wikipedia.org/wiki/WebSocket
You will need to run a daemon if you want to run it on your apache server.
Also take a look at:
http://kaazing.com/ if you wan't less hassle, but are willing to fork out some cash.
在 Windows 端，性能监视器是您应该研究的工具。
正如 Jared O '康纳说，你应该明确你想要监控哪种网络服务器。
On the Windows side, Perfmonance monitor is the tool you should investigate.
As Jared O'Connor said, you should precise what kind of web server you want to monitor.