记录请求并计时
记录 Web 应用程序的 http 请求(包括 ajax 请求)的最佳方法是什么,以便我稍后可以返回并查询“我想知道此请求发出了多少次,以及平均需要多长时间才能完成” ,或“显示前 5 个最高平均时间请求”
您是否会使用当前生产数据库中的单独数据库来记录这些内容,以防止所有这些插入导致 IO 减慢,或者这最终不会真正产生很大的影响?
您会批量处理请求然后推送到数据库,还是会为每个请求执行一次插入?
除了将每个请求处理程序包装在应用程序逻辑中之外,是否有更好的方法来添加此请求登录计时:
start = CurrentTime()
/* request handler code */
end = CurrentTime()
Insert(requestName, start, (end - start))
What is the best way to log http requests for a web application, including ajax requests, so that I can later go back and query "I want to know how many times this request was made, and how long it took to complete on average", or "show me the top 5 highest average time requests"
Would you use a separate database from the current production db to log these things to prevent all of those inserts causing IO slowdown, or does this end up not really making a big impact?
Would you bulk up requests and then push to the DB or would you do a single insert for each request?
Is there a better way to add this request logging in with timings besides wrapping each request handler in the application logic like:
start = CurrentTime()
/* request handler code */
end = CurrentTime()
Insert(requestName, start, (end - start))
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
您应该能够使用您的网络服务器日志来实现此目的。 Apache 和 IIS 都会记录捕获 URL、查询字符串、响应代码和持续时间。如果 AJAX 请求通过 HTTP POST 接收数据,您将需要更改 Web 服务器的配置以捕获该数据(如果该数据对您很重要)。那么我找到的用于日志分析的最佳工具是 Microsoft 的日志解析器,它可以很好地使用 SQL 语法处理大型文件来计算您所提出的各种问题的答案。
但是,如果您打算自己动手,请使用某种本地日志记录。并使用日志框架 - 例如 log4J - 它足够智能,可以缓冲磁盘写入以最大限度地减少 I /O,滚动日志文件,并删除旧文件。对于集群服务器来说,这是更具可扩展性的方法,否则每个服务器都会不断地访问数据库。如果要将数据放入表中,请将其设为批处理,例如每小时或每天一次。
You should be able to use your web server logs for this purpose. Apache and IIS logs both capture URL, query string, response code and duration. If the AJAX requests receive data via HTTP POST you will need to change the web server's configuration to capture that data if it's important to you. Then the best tool I've found for log analysis is Microsoft's Log Parser, which does a great job working with large files with a SQL syntax to calculate answers to the kinds of questions you're asking.
However, if you're intent on rolling your own, use some sort of local logging. And use a logging framework - such as log4J - which is smart enough to buffer out disk writes to minimize I/O, roll the log files, and delete old files. This is more scalable approach for clustered servers, otherwise each is hitting the database constantly. If you want to put the data in a table, make it a batch process, once an hour or day for instance.
似乎是 Google Analytics(分析)的绝佳用例(请参阅事件跟踪,特别是)。
如果这不是一个选项,请尽早考虑可扩展性:
不要从您所服务的服务器页面内进行登录,因为这最终可能会影响缓存。使用脚本或 1x1 图像来传递参数,并使其在单独的(非缓存)进程中运行。
如果您最终将硬盘驱动器设为基于数据库,请避免对其进行重击。使用基于内存的存储来存储统计数据,并定期将其内容保存到数据库中。 (回想一下,当 Google 最初打开它时,Google Analytics 不知所措。)
Seems like an excellent use-case for Google Analytics (see event tracking, in particular).
If it's not an option, think early about scalability:
Don't log from within the server page you're serving, as this can end up working against caching. Use a script or 1x1 image to toss in parameters, and have that operate in a separate (non-cached) process.
Avoid hammering your hard drive if you end up making it DB-based. Use memory-based storage to store stats as they come, and periodically persist its content into your database. (Recall that Google Analytics was overwhelmed when Google initially opened it.)
如果您的目标是简单地跟踪请求的时间并将其保存到数据库,但您不希望插入可能使您的请求陷入困境,那么您应该使用支持高速插入的数据库,例如 MongoDB。如果将连接字符串调整为 strict mode = off,则插入基本上是异步的。
这是一个很好的起点:
NoSQ LMovement、LINQ 和 MongoDB
如果您仅通过 ASP.NET MVC 执行此操作,那么您可以将其与自定义操作过滤器放在一起,并将其从控制器代码中取出,如下所示 出色地。我想您也可以使用自定义 ASP.NET 模块来实现这一点,并获取 IIS 7+ 中的所有请求(甚至非 ASP.NET)。
如果 NoSQL / MongoDB 不是您的菜,您可以使用 SqlCommand.BeginExecuteNonQuery 进行插入,而不是使其成为阻塞调用。如果您插入到与生产相同的数据库,则可能会导致一些争用。
If your goal is to simply track the timing of requests and save them to a DB but you don't want the inserts to potentially bog down you requests, then you should use a DB that supports high-speed inserts such as MongoDB. If you tweak the connection string to strict mode = off then the inserts are basically asynchronous.
Here's a good starting point:
The NoSQ LMovement, LINQ, and MongoDB
If you're doing this over ASP.NET MVC only, then you could put that together with a custom action filter and get it out of your controller code as well. I suppose you can also achieve that with a custom ASP.NET module and get all the requests (even non-ASP.NET) in IIS 7+.
If NoSQL / MongoDB isn't your thing, you can use SqlCommand.BeginExecuteNonQuery for inserts rather than making it a blocking call. If you insert to the same DB as production though you can cause some contention there.