调试“ mem_used_percent”在EC2实例上
在EC2实例上运行的AWS服务中,我们面临着一个奇特的问题。在CW指标中,“ mem_used_percent”逐渐上升,最终达到90%,随后是系统故障。我们已经验证了由于OOM错误而发生的故障,并且重新启动主机正在通过将“ mem_used_percent”下降到20%左右来修复它。
在对新运行的EC2实例进行调查时,我们看到在“ TOP”命令输出(按%MEM)中仅考虑了大约20%的RAM使用情况。我们无法使用未算出的物理内存实际上固定过程。
是否有更好的方法对EC2实例(Linux)进行内存使用分析,以总结到CW指标中的“ mem_used_percent”?
请让我知道是否需要其他细节。
谢谢!!
We are facing a peculiar issue in our AWS service running on EC2 instances. In CW metrics "mem_used_percent" is gradually going up with time and eventually goes to 90% followed by system failure. We have verified that failure is happening due to OOM error and restarting the hosts is fixing it by bringing the "mem_used_percent" down to around 20%.
While doing the investigation on new and old running EC2 instances, we are seeing that only around 20% of the RAM usage is accounted for in "top" command output(sort by %MEM). We are not able to actually pin-point the processes using rest of the unaccounted physical memory.
Is there a better way to do memory usage analysis on EC2 instances(Linux) that will sum up to "mem_used_percent" in CW metrics?
Please let me know if any other details are required.
Thanks!!
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
看一下“ Procstat插件”。它可能有助于您的调查
“ Procstat插件使您可以从各个流程中收集指标”。
https://docs.aws.amazon.com/amazoncloudwatch/latest/monitoring/cloudwatch-agent-procstat-procstat-process-metrics.html#cloudwatch-watch-watch-cloudwatch-ag agent-agent-procest-procest-procest-procest-procest-procest-procest-procest-metrics-collect,coll,Coll,Coll,Coll,Collected
Take a look at "procstat plugin". It might help your investigation
"The procstat plugin enables you to collect metrics from individual processes".
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-procstat-process-metrics.html#CloudWatch-Agent-procstat-process-metrics-collected