Windows下如何共享日志文件？

Question

我有几个不同的进程，我希望它们都记录到同一个文件。这些进程在Windows 7系统上运行。有些是 python 脚本，有些是 cmd 批处理文件。

在 Unix 下，你只需让每个人都以追加模式打开文件并写入即可。只要每个进程在单个消息中写入的字节数少于 PIPE_BUF 字节，就可以保证每个 write 调用不会与任何其他调用交错。

在 Windows 下有没有办法实现这一点？这种天真的类 Unix 方法失败了，因为默认情况下 Windows 不喜欢多个进程同时打开一个文件进行写入。

Answer 1

可以将多个批处理进程安全地写入单个日志文件。我对 Python 一无所知，但我想这个答案中的概念可以与 Python 集成。

Windows 最多允许一个进程在任何时间点打开一个特定文件进行写访问。这可用于实现基于文件的锁定机制，以保证事件在多个进程之间序列化。请参阅 https://stackoverflow.com/a/9048097/1012053 和 http://www.dostips.com/forum/viewtopic.php?p=12454 获取一些示例。

由于您要做的只是写入日志，因此您可以使用日志文件本身作为锁。日志操作封装在一个子例程中，该子例程尝试以附加模式打开日志文件。如果打开失败，例程将循环返回并重试。一旦打开成功，日志就会被写入然后关闭，并且例程返回给调用者。该例程执行传递给它的任何命令，并且例程中写入 stdout 的任何内容都将重定向到日志。

下面是一个测试批处理脚本，它创建 5 个子进程，每个子进程写入日志文件 20 次。写入是安全交错的。

@echo off
setlocal
if "%~1" neq "" goto :test

:: Initialize
set log="myLog.log"
2>nul del %log%
2>nul del "test*.marker"
set procCount=5
set testCount=10

:: Launch %procCount% processes that write to the same log
for /l %%n in (1 1 %procCount%) do start "" /b "%~f0" %%n

:wait for child processes to finish
2>nul dir /b "test*.marker" | find /c "test" | >nul findstr /x "%procCount%" || goto :wait

:: Verify log results
for /l %%n in (1 1 %procCount%) do (
  <nul set /p "=Proc %%n log count = "
  find /c "Proc %%n: " <%log%
)

:: Cleanup
del "test*.marker"
exit /b

==============================================================================
:: code below is the process that writes to the log file

:test
set instance=%1
for /l %%n in (1 1 %testCount%) do (
  call :log echo Proc %instance% says hello!
  call :log dir "%~f0"
)
echo done >"test%1.marker"
exit

:log command args...
2>nul (
  >>%log% (
    echo ***********************************************************
    echo Proc %instance%: %date% %time%
    %*
    (call ) %= This odd syntax guarantees the inner block ends with success  =%
            %= We only want to loop back and try again if redirection failed =%
  )
) || goto :log
exit /b

以下输出表明每个进程的所有 20 次写入均成功。

Proc 1 log count = 20
Proc 2 log count = 20
Proc 3 log count = 20
Proc 4 log count = 20
Proc 5 log count = 20

您可以打开生成的“myLog.log”文件以查看写入是如何安全交错的。但输出太大，无法在此发布。

通过修改 :log 例程使其在失败时不会重试，很容易证明来自多个进程的同时写入可能会失败。

:log command args...
>>%log% (
  echo ***********************************************************
  echo Proc %instance%: %date% %time%
  %*
)
exit /b

以下是“破坏” :log 例程后的一些示例结果

The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
Proc 1 log count = 12
Proc 2 log count = 16
Proc 3 log count = 13
Proc 4 log count = 18
Proc 5 log count = 14

Answer 2

你可以尝试一下这个Python模块：
http://pypi.python.org/pypi/ConcurrentLogHandler

它提供了一个直接替代RotatingFileHandler 允许多个进程同时记录到单个文件，而不会删除或破坏日志事件。

我没有使用过它，但我在阅读相关错误时发现了它（问题 4749 ）在Python中。

如果您实现自己的代码而不是使用该模块来执行此操作，请确保您阅读了该错误！

您可以使用输出重定向 在 Windows 上就像在 Bash 中一样。将批处理文件的输出通过管道传输到通过 ConcurrentLogHandler 进行记录的 Python 脚本。