根据系统资源使用情况监控和终止Python脚本
监视 python 脚本正在使用的系统资源并在资源使用超过某些预定值时终止它的“正确”或“最佳”方法是什么。就我而言，内存使用情况值得关注。尽管我愿意接受建议，但我并不是在问如何衡量系统资源的使用情况。
作为一个简单的例子，我们假设我有一个函数，可以找到小于某个大数的素数，并根据某些条件将它们添加到列表中。我事先不知道有多少素数会满足条件，所以如果我使用了太多的系统内存（比如说 8GB），我要确保终止该函数。 我知道有一些方法可以监控 python 对象的大小。我不知道的是，监视列表大小和退出的正确方法是在 prime 函数循环中包含一个大小测试，如果超过 8GB 或存在“外部”（我的意思是外部），则退出在循环外部但仍在 python 脚本之内或一部分）的方式来监视和退出。
就我而言，我在 Mac 上运行，但一般都会问这个问题。
What is the "right" or "best" way to monitor the system resources a python script is using and terminate it if the resource use exceeds some predetermined values. In my case memory usage is of concern. I am not asking how to measure the system resource use although I am open to suggestions.
As a simple example, let's assume I have a function that finds prime numbers less than some large number and adds them to a list based on some condition. I don't know ahead of time how many prime numbers will satisfy the condition so I what to be sure to terminate the function if I use up to much system memory (8gb lets say).
I know that there are ways to monitor the size of python objects. What I don't know is the proper way to monitor the size of the list and exit is to just include a size test in the prime function loop and exit if it exceeds 8gb or if there is an "external" (by external I mean external to the loop but still within or part of the python script) way to monitor and exit.
In my case I am running on a mac but am asking the question in general.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
在类 Unix 系统上，监视任何进程的一种有用的“外部”方法是 ulimit 命令（您没有明确是否要在 Windows 中运行，其中
ulimit不存在，其他方法可能存在，但我不知道它们；-）。如果您正在考虑在自己的 Python 程序中执行此类控制，只需更改相关函数来检查它附加到列表中的每个对象的大小（并保留运行总计），并在运行总计达到或超过时返回阈值（您可以将其作为额外参数传递给相关函数）。
编辑：OP在评论中澄清，他们希望将监控放在可能放置的最糟糕的地方——在前面的段落中，我提到了在流程之外很容易，在流程内部也很容易函数，但OP希望它“在中间”；-)。
最不糟糕的方法可能是使用“看门狗线程”——无限循环中的一个单独的守护线程，每隔 X 秒检查进程的资源消耗（例如使用 resource.getrusage，如果在类 Unix 机器上 -- 同样，如果在 Windows 上，需要其他东西），并且，如果消耗超过所需的限制，则尝试使用 thread.interrupt_main。当然，这是万无一失的失败：周期
X（就像在所有“轮询”的情况下）必须足够低，以同时阻止失控的进程，但又足够高，以免减慢进程几乎爬行。另外，主线程（唯一可以像这样中断的线程）可能会阻塞异常（在这种情况下，看门狗线程可能会尝试使用越来越严重的“向这个进程发出信号”，一直到SIGKILL，永远无法被阻止或拦截的杀手信号）。因此，这种中间方法比 ulimit 命令需要更多工作，更脆弱，并且没有实质性的附加价值。但是，如果您想将监视“放在进程内部，但在资源消耗功能之外”，没有任何优点，需要大量工作，并且有我提到的其他缺点，那么这就是实现的方法。
On Unix-like system, a useful "external" way to monitor any process is the ulimit command (you don't clarify whether you want instead to run in Windows, where
ulimitdoesn't exist and other approaches may, but I don't know them;-).If you're thinking about performing such controls inside your own Python programs, just change the function in question to check the size of each object it's appending to the list (and keep a running total) and return when the running total reaches or exceeds a threshold (which you could pass as an extra parameter to the function in question).
Edit: the OP has clarified in a comment that they want the monitoring in the very worst place it could possibly be placed -- in the previous paragraphs, I mentioned how it's easy outside of the process, easy inside the function, but the OP wants it "smack in the middle";-).
Least-bad way is probably with a "watchdog thread" -- a separate daemon thread in an infinite loop which, every
Xseconds, checks the process's resource consumption (e.g. with resource.getrusage, if on Unix-like machines -- again, if on Windows, something else is needed instead) and, if that consumption exceeds the desired limits, attempts to kill the main thread with thread.interrupt_main. Of course, this is fail from foolproof: the periodicityX(like in all cases of "polling") must be low enough to stop a runaway process in the meantime, but high enough to not slow the process down to a crawl. Plus, the main thread (the only one that can be interrupted like this) might be blocking exceptions (in which case the watchdog thread might perhaps try with "signals to this very process" of growing severity, all the way up toSIGKILL, the killer-signal that can never be blocked or intercepted).So, this intermediate approach is a lot more work than the
ulimitcommand, is more fragile, and has no substantial added value. But, if you want to put the monitoring "inside the process but outside the resource-consuming function", with no advantages, lots of work, and the other disadvantages I've mentioned, this is the way to do it.resource.getrusage()（在特定的ru_idrss）可以为您提供当前python解释器的资源使用情况，您可以将其用作停止处理的哨兵。resource.getrusage()(in particularru_idrss) can give you the resource usage of the currentpythoninterpreter, which you can use as a sentinel to stop processing.