popen.communate提高了ValueError:' i/o操作在封闭文件上'

发布于 2025-02-11 21:21:18 字数 2667 浏览 1 评论 0 原文

运行代码时,我会遇到以下错误。

    Uncaught exception <class 'ValueError'>: I/O operation on closed file.Traceback (most recent call last):
    File "/home/gidumah/miniconda/envs/ytune/lib/python3.7/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/home/gidumah/miniconda/envs/ytune/lib/python3.7/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/home/gidumah/ytopt/ytopt/ytopt/search/ambs.py", line 128, in <module>
        search.main()
      File "/home/gidumah/ytopt/ytopt/ytopt/search/ambs.py", line 105, in main
        results = list(self.evaluator.get_finished_evals())
      File "/home/gidumah/ytopt/ytopt/ytopt/evaluator/evaluate.py", line 200, in get_finished_evals
        y = future.result()
      File "/home/gidumah/ytopt/ytopt/ytopt/evaluator/subprocess_evaluator.py", line 41, in result
        stdout, stderr_data = self.proc.communicate()
      File "/home/gidumah/miniconda/envs/ytune/lib/python3.7/subprocess.py", line 951, in communicate
        stdout = self.stdout.read()
    ValueError: I/O operation on closed file.

我正在运行的代码是 subprocess_evaluator.py ,如下:

class PopenFuture:
    FAIL_RETURN_VALUE = Evaluator.FAIL_RETURN_VALUE

    def __init__(self, args, parse_fxn):
        self.proc = subprocess.Popen(args, shell=True, stdout=subprocess.PIPE,
                                     stderr=subprocess.STDOUT, encoding='utf-8')

        self._state = 'active'
        self._result = None
        self._parse = parse_fxn

    def _poll(self):
        if not self._state == 'active':
            return
        retcode = self.proc.poll()
        if retcode is None:
            self._state = 'active'
            stdout, stderr_data = self.proc.communicate()
            tmp_res = self._parse(stdout)
            if tmp_res != sys.float_info.max:
                self._result = tmp_res
        elif retcode == 0:
            self._state = 'done'
        else:
            self._state = 'failed'

    def result(self):
        if self._result is not None:
            return self._result
        self.proc.wait()
        stdout, stderr_data = self.proc.communicate()
        if self.done:
            self._result = self._parse(stdout)
        else:
            self._result = self.FAIL_RETURN_VALUE
            logger.error(f"Eval failed: {stdout}")
        ####
        # if stdout:
        #     print (stdout)#.split('\n')[:-2])#)[:-1])
        ####   
        return self._result

它说该错误在第41行中,以 stdout,stderr_data = self.proc.communate() stdout。 /code>

我在通信通话中没有做正确的事情吗?

I get the following error when I run my code.

    Uncaught exception <class 'ValueError'>: I/O operation on closed file.Traceback (most recent call last):
    File "/home/gidumah/miniconda/envs/ytune/lib/python3.7/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/home/gidumah/miniconda/envs/ytune/lib/python3.7/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/home/gidumah/ytopt/ytopt/ytopt/search/ambs.py", line 128, in <module>
        search.main()
      File "/home/gidumah/ytopt/ytopt/ytopt/search/ambs.py", line 105, in main
        results = list(self.evaluator.get_finished_evals())
      File "/home/gidumah/ytopt/ytopt/ytopt/evaluator/evaluate.py", line 200, in get_finished_evals
        y = future.result()
      File "/home/gidumah/ytopt/ytopt/ytopt/evaluator/subprocess_evaluator.py", line 41, in result
        stdout, stderr_data = self.proc.communicate()
      File "/home/gidumah/miniconda/envs/ytune/lib/python3.7/subprocess.py", line 951, in communicate
        stdout = self.stdout.read()
    ValueError: I/O operation on closed file.

The code I was running is subprocess_evaluator.py, which is given below:

class PopenFuture:
    FAIL_RETURN_VALUE = Evaluator.FAIL_RETURN_VALUE

    def __init__(self, args, parse_fxn):
        self.proc = subprocess.Popen(args, shell=True, stdout=subprocess.PIPE,
                                     stderr=subprocess.STDOUT, encoding='utf-8')

        self._state = 'active'
        self._result = None
        self._parse = parse_fxn

    def _poll(self):
        if not self._state == 'active':
            return
        retcode = self.proc.poll()
        if retcode is None:
            self._state = 'active'
            stdout, stderr_data = self.proc.communicate()
            tmp_res = self._parse(stdout)
            if tmp_res != sys.float_info.max:
                self._result = tmp_res
        elif retcode == 0:
            self._state = 'done'
        else:
            self._state = 'failed'

    def result(self):
        if self._result is not None:
            return self._result
        self.proc.wait()
        stdout, stderr_data = self.proc.communicate()
        if self.done:
            self._result = self._parse(stdout)
        else:
            self._result = self.FAIL_RETURN_VALUE
            logger.error(f"Eval failed: {stdout}")
        ####
        # if stdout:
        #     print (stdout)#.split('\n')[:-2])#)[:-1])
        ####   
        return self._result

It says the error is in line 41, which is given as stdout, stderr_data = self.proc.communicate()

Is there something I am not doing right with the communicate call?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

删除会话 2025-02-18 21:21:18

popen。交流(input = none,timeout = none)

与过程互动:
将数据发送到STDIN。从Stdout和Stderr读取数据,直到
到达文件结束。等待过程终止并设置
返回代码属性。可选输入参数应为数据
发送到子进程,或不发送,如果不应将数据发送到
孩子。如果以文本模式打开流,则必须是字符串。
否则,必须是字节。

请注意,它等待该过程终止。因此, popen.communate 不能多次运行。 popen.communate 的上下文中的错误意味着该程序已经终止了该过程并关闭了管道,并在第二通信呼叫时提高了例外。

看来您正在使用 ytopt 库。请检查您是否正确使用代码,请注意,在几个不同的位置调用 communicate(),有时会通过valueerrors传递,但有时不会传递。另外,请查看Thomas Wouters 解决方案解决问题:

如果您想使用通信,则必须重新启动该过程
在循环中,或给它一个单字符串,该字符串是来自
发电机。如果您想进行流媒体通信,请发送数据位
一点一点地,您必须不使用通信。相反,您需要
从P.Stdout和P.Stderr阅读时写信给P.Stdin。正在做
这很棘手,因为您无法确定哪个输出是由哪个输出引起的
输入,并且因为您可以轻松地陷入僵局。有
可以为您提供帮助的第三方库,例如扭曲。

如果您想进行互动执行此操作,请发送一些数据,然后
在发送更多数据之前等待和处理结果
变得更加努力。您可能应该使用第三方图书馆
为此。

当然,如果您可以摆脱内部的过程
循环,这要容易得多:

  cmd = ['Intersected','-a','stdin','-b',床file]
要在my_entry_generator中输入:
    p = subprocess.popen(cmd,stdin = subprocess.pipe,stdout = subprocess.pipe,stderr = subprocess.pipe)
    输出= p.communate(input ='\ t'.join(entry) +'\ n')[0]
    打印输出
 

Accoring to the documentation:

Popen.communicate(input=None, timeout=None)

Interact with process:
Send data to stdin. Read data from stdout and stderr, until
end-of-file is reached. Wait for process to terminate and set the
returncode attribute. The optional input argument should be data to be
sent to the child process, or None, if no data should be sent to the
child. If streams were opened in text mode, input must be a string.
Otherwise, it must be bytes.

Note that it waits for the process to terminate. Therefore Popen.communicate cannot be run more than once. The error in the context of Popen.communicate means that the program already terminated the process and closed the pipes, and raised the exception upon second communicate call.

It seems that you are using code of the ytopt library. Please check if you use the code correctly, note that it calls communicate() in few different places, sometimes with passing on ValueErrors but sometimes not. Also, check out Thomas Wouters solution to the problem:

If you want to use communicate, you have to either restart the process
in the loop, or give it a single string that is all the input from the
generator. If you want to do streaming communication, sending data bit
by bit, then you have to not use communicate. Instead, you would need
to write to p.stdin while reading from p.stdout and p.stderr. Doing
this is tricky, because you can't tell which output is caused by which
input, and because you can easily run into deadlocks. There are
third-party libraries that can help you with this, like Twisted.

If you want to do this interactively, sending some data and then
waiting for and processing the result before sending more data, things
get even harder. You should probably use a third-party library like
pexpect for that.

Of course, if you can get away with just starting the process inside
the loop, that would be a lot easier:

cmd = ['intersectBed', '-a', 'stdin', '-b', bedfile]
for entry in my_entry_generator:
    p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    output = p.communicate(input='\t'.join(entry) + '\n')[0]
    print output
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文