Boost C++ Libraries Home Libraries People FAQ More

PrevUpHomeNext

Why do I get a fatal application exit with FATAL EXCEPTION: Failed to read all buffers when I read a file?

This is actually a safety checkpoint for your code: in complex, multi-process concurrent reading and writing of the same file, it is extremely difficult to coordinate changing file lengths with i/o in a way which doesn't introduce race conditions OR unacceptably low performance. AFIO therefore doesn't even try[14] and simply requires you the programmer to ALWAYS do i/o, whether reading or writing, within the extent of a file. In other words, if you're going to read 100 bytes from offset 100 in a file, that file better be at least 200 bytes long or it's going to fail with a fatal application exit.

This will probably seem harsh to anyone using AFIO for the first time, because the following naive code will fatal exit the application if foo.txt is not 1024 bytes or longer:

char input[1024];
auto file_opened = file("foo.txt");
read(file_opened, input, 0);

With synchronous i/o a read of 1024 bytes will read up to 1024 bytes, returning the amount actually read via some mechanism. With AFIO, either you read all 1024 bytes or you read nothing, in which case a normal exception is thrown with whatever error the operating system returned. If a partial read happens, then AFIO fatal exits the application with the above message as it is probably a logic error in your code.

You may now wonder how to easily not exceed file extents during i/o: for writing, see async_truncate() to ensure a file's size before writing, or else open a file for append-only in which case all writes atomically occur at the end of the file. For reading, the following code is suggested:

char input[1024];
// Open the file synchronously
auto file_opened = file("foo.txt");
// Fetch ONLY the size metadata. Blocks because it's synchronous!
directory_entry de = file_opened->direntry(metadata_flags::size);
// Read now we know the file size
read(file_opened, (void *) input,
     (size_t) de.st_size(),  // doesn't block, as size was fetched before.
     0);

If you're going to read many files from the same directory, it is far faster to open a handle to the containing directory and using enumerate to fetch the metadata asynchronously instead of using direntry() which is synchronous:

char input[1024];
// Schedule enumerating the containing directory, but only for foo.txt
auto dir_opened = async_dir("");  // "" means current directory in AFIO
auto file_enumed =
async_enumerate(dir_opened, metadata_flags::size, 2, true, "foo.txt");
// Schedule in parallel opening the file
auto file_opened = async_file(dir_opened, "foo.txt");
auto file_read = file_enumed.then(
[&input](future<std::pair<std::vector<directory_entry>, bool>> &f)
{
  // Get the directory_entry for the first result
  directory_entry &de = f.get().first.front();
  // Schedule a file read once we know the file size
  return async_read(f, (void *) input,
                    (size_t) de.st_size(),  // won't block
                    0);
});
file_read.get();


[14] AFIO will try to provide a synchronised, accurate file extent after fast portable file locking support has been added, but until then no.


PrevUpHomeNext