Let's say that your program is writing information to a file. How does the disk recover your file if there is a system crash?

Question

And that system crash happens while your file is being written to by your program?

Also, is it the case that when your program writes to the file, it actually isn't writing directly to disk, but rather to memory instead?

Because it just seems like it would be more effective for the system to transfer the written data to memory, and then when it's completely written to transfer things back to the disk.

In any case, how does the disk recover data whether your program writes directly to disk, or whether the finished file is in the process of being copied over by the system to various locations on the disk?

Answer 1

Because it just seems like it would be more effective for the system to transfer the written data to memory, and then when it's completely written to transfer things back to the disk.

In modern operating systems, file access is buffered. Your program writes to a part of memory that's maintained by the OS, and when that area gets full, or when you close the file, the buffer is written to disk. This way, you can write several gigabytes of random data into a file if you want, but you don't consume that much memory while you're doing it. At the same time, the disk subsystem is free to do other things while your program is writing a few bytes at a time.

In any case, how does the disk recover data whether your program writes directly to disk, or whether the finished file is in the process of being copied over by the system to various locations on the disk?

Depends on the operating system and on the particular file system. Often, the data is written into a new file on disk. When your program closes the file, the buffer is flushed and the directory is updated to point to the new file instead of the old one, which can subsequently be deleted. This avoids data corruption if the system crashes during writing -- changing the directory at the last moment creates the illusion of "atomic" writes.

You might want to read about journaled file systems and transactional file systems.

Other strategies are also possible. Entire textbooks are available on file systems and fault tolerance.

Finally, sometimes the file system can't recover your file. Sometimes it can't recover at all. The strategies I mentioned above are responses to the kinds of problems you're asking about, and as far as I know they're pretty effective at keeping the disk in a consistent state. Older file systems weren't so good at that, and if you happened to lose power just when the disk was being written, the file or the whole disk could be compromised. There used to be a large market for disk repair utilities such as Norton Disk Doctor. That market seems to have waned with more reliable OS implementations, but some of those products are still available.

Answer 2

Most modern file systems are journaled. This means that there is a journal of changes to the file/filesystem. On crashes, the last entry can be rolled back if it is not marked as completed; or if there is sufficient information, can be enacted.

Wikipedia has this:

To prevent this [data loss on crash], a journaled file system allocates a special area—the journal—in which it records the changes it will make, ahead of time. After a crash, recovery simply involves reading the journal from the file system and replaying changes from this journal until the file system is consistent again. The changes are thus said to be atomic (or indivisible) in that they either: succeed (succeeded originally or are replayed completely during recovery), or are not replayed at all (are skipped because they had not yet been completely written to the journal before the crash occurred).

In windows (vista and beyond), there is a technology called Volume Shadow Copy (VSS), which makes the file system act in a "copy-on write" manner, meaning when you write to a file, that old file is not being destroyed, and is in fact maintained intact. This allows restoring after crashes, but more importantly, it also allows access to the last stable version of the file in the event of a file lock and a process (such as a backup service) needs to read that file.