从损坏的文件中恢复的策略?
我和我的同事正在尝试实现一种机制来从嵌入式设备上的损坏文件中恢复。
在某些情况下可能会发生这种情况,例如用户在写入文件时取下电池。
Orz,但现在我们只有一个想法:
- 创建重复的备份文件,如果危险文件 I/O 未正确完成,则将它们复制回来。
这有点愚蠢,好像备份文件也坏了,我们就死定了。
您对此有什么建议或好的文章吗?
提前致谢。
Me and my colleague are trying to implement a mechanism to get recovery from broken files on an embedded equipment.
This could be happened during certain circumstances, e.g. user takes off the battery during file writing.
Orz, but now we have just one idea:
- Create duplicated backup files, and copy them back if dangerous file i/o is not finished properly.
This is kind of stupid, as if the backup files also broken, we are just dead.
Do you have any suggestions or good articles on this?
Thanks in advance.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
阅读数据库日志记录和数据库日志文件。
数据库(如 Oracle)具有非常非常强大的文件写入功能。 实际上并没有使用Oracle。 使用他们的设计模式。 设计模式是这样的。 您可以借用这些想法,而无需实际使用实际产品。
您的事务(即插入)将获取要更新的块。 通常这是在内存缓存中,如果没有,则从磁盘读取到内存缓存。
“前映像”(或回滚段)副本由您要写入的块组成。
“前映像
您更改缓存副本,写入日志条目,并对数据库写入进行排队。
您
您提交更改,这使得缓存更改对其他事务可见。
在某个时刻,数据库编写器将完成数据库文件更改。
日志是一个简单的循环队列文件——记录只是变化的历史,几乎没有结构。 它可以在多个设备上复制。
DB 文件的结构更为复杂。 他们有一个“交易编号”——总体交易的简单顺序计数。 这是在块中编码(两种不同的方式)并写入控制文件。
优秀的 DBA 可确保控制文件在设备之间进行复制。
当 Oracle 启动时,它会检查控制文件以找出哪一个可能是正确的。 其他人可能已损坏。 Oracle 检查数据库文件以查看哪些文件与控制文件匹配。 它检查日志以查看是否需要应用交易以使文件达到正确的交易编号。
当然,如果它在写入所有日志副本时崩溃,则该事务将会丢失——对此无能为力。 但是,如果在写入日志条目后崩溃,它可能会完全恢复而不会出现任何问题。
如果您丢失介质并恢复备份,则日志文件有可能应用于恢复的备份文件并将其更新为最新。 否则,必须重播旧的日志文件才能使其更新。
Read up on database logging and database journal files.
A database (like Oracle) has very, very robust file writing. Do not actually use Oracle. Use their design pattern. The design pattern goes something like this. You can borrow these ideas without actually using the actual product.
Your transaction (i.e., Insert) will fetch the block to be updated. Usually this is in memory cache, if not, it is read from disk to memory cache.
A "before image" (or rollback segment) copy is made of the block you're about to write.
You change the cache copy, write a journal entry, and queue up a DB write.
You commit the change, which makes the cache change visible to other transactions.
At some point, the DB writer will finalize the DB file change.
The journal is a simple circular queue file -- the records are just a history of changes with little structure to them. It can be replicated on multiple devices.
The DB files are more complex structures. They have a "transaction number" -- a simple sequential count of overall transactions. This is encoded in the block (two different ways) as well as written to the control file.
A good DBA assures that the control file is replicated across devices.
When Oracle starts up, it checks the control file(s) to find which one is likely to be correct. Others may be corrupted. Oracle checks the DB files to see which match the control file. It checks the journal to see if transactions need to be applied to get the files up to the correct transaction number.
Of course, if it crashes while writing all of the journal copies, that transaction will be lost -- not much can be done about that. However, if it crashes after the journal entry is written, it will probably recover cleanly with no problems.
If you lose media, and recover a backup, there's a chance that the journal file can be applied to the recovered backup file and bring it up to date. Otherwise, old journal files have to be replayed to get it up to date.
取决于哪个操作系统等,但在大多数情况下,您可以做的是复制到临时文件名,并作为最后一步将文件重命名为正确的名称。
这意味着(WOOPS)潜在 S****p 的机会之窗仅限于重命名发生的时间间隔内。
如果操作系统支持良好的目录结构并且您可以智能地布置文件,则可以通过将新文件复制到临时目录并重命名该目录来进一步完善此结构,以便 WOOPS 成为“重命名目标以保存”和“重命名临时”之间的间隔达到目标”。
如果操作系统支持软链接目录,那么您可以“ln -s target temp”,这会变得更好。 在大多数操作系统上,替换软链接将是一个“原子”操作,无论是否工作,都不会出现任何混乱的中间状态。
所有这些选项都取决于是否有足够的存储空间来在文件系统上保留完整的旧副本和新副本。
Depends on which OS etc. etc. but in most cases what you can do is copy to a temporary file name and as the last final step rename the files to the correct name.
This means the (WOOPS) Window of Opertunity Of Potential S****p is confined to the interval when the renames take place.
If the OS supports a nice directory structure and you lay out the files intelligently you can further refine this by copying the new files to a temp directory and renaming the directory so the WOOPS becomes the interval between "rename target to save" and "rename temp to target".
This gets even better if the OS supports Soft link directories then you can "ln -s target temp". On most OSes replacing a softlink will be an "atomic" operation which will work or not work without any messy halfway states.
All these options depend on having enough storage to keep a complete old and new copy on the file system.