当前位置：文江博客话题详情

在非事务性文件系统中实现原子文件写入

发布于 2025-01-01 00:01:08 字数 1451 浏览 2 评论 0原文

许多常见的文件系统不提供原子操作，但在某些场景下以原子方式写入文件非常重要。我试图想出一个解决这个问题的方法。

我做了以下假设：

使用的文件系统支持 inode 级别的原子操作（例如 NTFS）。这意味着移动和删除是原子的。
只有程序本身才能访问这些文件。
该程序一次只有 1 个实例，并且以单线程方式运行。
为了简单起见，每次都会写入整个文件内容（即截断写入）。

这就留下了以下问题：在写入文件时，程序可能会被中断，并且文件将只剩下部分内容要写入。

我建议执行以下过程：

将新内容写入临时文件新建
将原始文件原始移动到临时位置备份
移动新建到原始
删除备份

新和备份文件与原始文件区分开来> 文件（例如，它们可以有不同的前缀，或者可以位于单独的文件中）同一卷上的目录）。同时，它们的名称应直接映射到相应的原始（例如简单地使用相同的文件名）。

然而，这并没有使操作成为原子操作。该过程可能会在步骤 1、2、3 或 4 中中断：

留下可能不完整的新。
移动是原子的，但目标文件现在丢失。新建和备份均存在且完整。
移动是原子的，但有一个未使用的备份。原始被新内容替换
删除是原子的。

使用之前的假设 2 和 3，程序必须在崩溃后重新启动。在启动过程中，它应该执行以下恢复检查：

如果New存在，但Backup不存在，则我们在步骤1中或之后崩溃。删除New > 因为它可能不完整。
如果New存在且Backup也存在，则我们在步骤2后崩溃。继续步骤3。
如果Backup存在但New em> 也没有，我们在步骤 3 后崩溃了。继续步骤 4。

恢复过程本身仅使用原子操作，在被中断后将简单地从中断处继续。

我相信这个想法可以确保单个程序的原子写入。这些问题仍然存在：

当使用同一程序的多个实例时，恢复过程会干扰另一个程序中当前正在进行的文件写入。
只读但从不写的外部程序通常会得到正确的结果，但如果同时对请求的条目有写操作，它们可能会错误地找不到条目。

这些问题（之前的假设已排除）可以通过使用策略来解决（例如，检查其他实例，并拒绝其他用户的目录访问）。

最后，我的问题是：这是否有意义，或者过程中是否存在缺陷？是否存在任何问题阻碍这种方法在实践中使用？

原文

Many common filesystems do not offer atomic operations, yet writing files in an atomic manner is very important in certain scenarios. I tried to come up with a solution for this problem.

I made the following assumptions:

The filesystem in use supports atomic operations at inode level (for instance, NTFS). This means that move and delete are atomic.
Only the program itself accesses the files.
There is only 1 instance of the program at a time and it acts in a single-threaded manner.
For simplicity, the whole file content is written each time (i.e. truncate-write).

This leaves the following problem: While writing a file, the program could be interrupted and the file would be left with only a part of the content to write.

I propose the following process:

Write new content to a temporary file New
Move the original file Original to a temporary location Backup
Move New to Original
Delete Backup

New and Backup files are distinguishable from Original files (for instance, they could be prefixed differently, or could be in a separate directory on the same volume). At the same time, their name should map directly to the corresponding Original (for instance by simply using the same file name).

This, however, does not make the operation atomic yet. The process could be interrupted steps 1, 2, 3 or 4:

Leaves a potentially incomplete New.
Move is atomic, but the target file is now missing. Both New and Backup exist and are complete.
Move is atomic, but there is an unused Backup. The Original was replaced by the New content
Deletion is atomic.

Using assumptions 2 and 3 from earlier, the program has to be restarted after a crash. During the startup process, it should perform these recovery checks:

If New exists but Backup does not, we crashed in or after step 1. Delete New since it could be incomplete.
If New exists and Backup does too, we crashed after step 2. Continue with step 3.
If Backup exists but New does not, too, we crashed after step 3. Continue with step 4.

The recovery process itself, only using atomic operations, will simply continue where it left off after being interrupted.

I believe this idea ensures atomic writes for a single program. These issues exist still:

When using multiple instances of the same program, there is an interference of the recovery process with currently ongoing file writes in the other program.
Outside programs that only read but never write will usually get the correct result, but if there is a write operation on the requested entry at the same time, they may incorrectly find no entry.

Those issues (which are excluded by the assumptions earlier) could be solved via usage policy (for instance, check for other instances, and deny directory access to other users).

Finally, my question: Did that make sense, or is there a flaw in the process? Are there any issues that prevent this approach from being used in practice?

分享到QQ

分享到微博