OSError：[Errno 24] 开发环境中 Google App Engine 任务队列打开的文件过多
我正在测试一个 Google App Engine 应用程序，我开始使用任务队列来处理批处理作业。我有一个本地作业，任务队列中似乎有许多任务，这些任务似乎在我的文件系统中创建了许多对象。需要明确的是：我没有创建任何文件，应用程序服务器似乎正在这样做。
我注意到在我的开发环境中创建大量任务（2000+）时，在某些时候我的作业开始失败并出现以下错误：
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> -->
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> --> -->
</font> </font> </font> </script> </object> </blockquote> </pre>
</table> </table> </table> </table> </table> </font> </font> </font><pre>Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 3211, in _HandleRequest
self._Dispatch(dispatcher, self.rfile, outfile, env_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 3135, in _Dispatch
'request.')
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/tempfile.py", line 302, in mkstemp
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/tempfile.py", line 236, in _mkstemp_inner
OSError: [Errno 24] Too many open files: '/var/folders/rm/rm1N57BDHNCyQUT2mQRTX++++TI/-Tmp-/request.QKY1gF.tmp'
</pre>
我使用的是 OS X 10.6.5，我猜这与我的操作系统 - 虽然我不太明白为什么 dev_appserver.py 会打开这么多文件描述符。即使它为每个任务创建一个，速率限制不会阻止此问题的发生吗？
我认为这是一个我只会在开发中看到的问题，但我想问是否有其他人看到过它。
I'm testing a Google App Engine application where I've started using task queues to process a batch job. I have a local job that seem a number of tasks in a task queue which seems to create a number of objects in my file system. To be clear: I'm not creating any files, the app server seems to be doing so.
I've noticed when creating large numbers of tasks (2000+) in my development environment that at some point my jobs start failing with the following error:
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> -->
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> --> -->
</font> </font> </font> </script> </object> </blockquote> </pre>
</table> </table> </table> </table> </table> </font> </font> </font><pre>Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 3211, in _HandleRequest
self._Dispatch(dispatcher, self.rfile, outfile, env_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 3135, in _Dispatch
'request.')
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/tempfile.py", line 302, in mkstemp
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/tempfile.py", line 236, in _mkstemp_inner
OSError: [Errno 24] Too many open files: '/var/folders/rm/rm1N57BDHNCyQUT2mQRTX++++TI/-Tmp-/request.QKY1gF.tmp'
</pre>
I'm on OS X 10.6.5 and I'm guessing it's something to do with my OS - though I can't quite wrap my head around why dev_appserver.py would open up so many file descriptors. Even if it's creating one per task won't the rate limiting prevent this problem from occurring?
I assume this is a problem I'll only see in dev, but I wanted to ask if anyone else has seen it.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
大卫是对的。您无法在本地文件系统中创建文件。
但是您得到的错误是由涉及 dev_appserver 中的任务的错误引起的。一旦出现错误，请尝试运行 lsof，您将看到开发服务器有一堆数据存储文件的句柄。
切换到使用 sqlite 后端这个问题就会消失。不要忘记，当您切换时，您将需要第一次清除数据存储。
David is correct. You can't create files in the local file system.
But the error your getting is caused by a a bug involving tasks in the dev_appserver. Once you get the error, try running
lsof, you'll see that the dev server has a bunch of handles to the datastore file.Switch to using the sqlite backend this problem will go away. Do not forget, when you switch you will need to clear the datastore the first time.
遗憾的是，在 App Engine 上，您无法在远程文件系统上创建任何文件（请参阅 文档的“沙盒”部分）。
如果您需要创建“文件”，则可以使用 数据存储区实体上的
BlobProperty（如果您的文件小于 1MB）或者可能是 Blobstore 用于较大的文件（取决于您的要求）。Unfortunately, on app engine you cannot create any files on the remote file system (see the "Sandbox" section of the documentation).
If you need to create "files" you might be able to use a
BlobPropertyon a datastore entity (if your files are smaller than 1MB) or perhaps the Blobstore for larger files (depending on your requirements).