Schedule a batch of asynchronous zeroing and deallocations of physical storage ("hole punching") after preceding operations.

Description

Most extent based filing systems provide an optimised way of zeroing parts of a file by deallocating the storage backing those regions, and marking those regions as unwritten instead of actually writing zero bytes to storage. They appear as zeroes to anything reading those ranges, and have the big advantage of not consuming any actual physical storage. On Windows, extent deallocation writes zeros for ordinary files and only actually deallocates physical storage if the file is sparse or compressed (note that AFIO by default creates sparse files where possible, and converts any file opened for writing to a sparse file). For your information, deallocation on NTFS is on a 64Kb granularity, but the zeros are written at a byte granularity. On Linux, an attempt is made to use FALLOC_FL_PUNCH_HOLE which if it fails then a write of zeros corresponding to the same ranges is made instead. On FreeBSD, long runs of zeros are automatically detected and eliminated on physical storage, and so zeros are simply written. On OS X, there is no formal hole punching API that we are aware of, and so zeros are simply written.

Synopsis

future dispatcher::zero(const std::vector< future<>> & ops, const std::vector< std::vector< std::pair< off_t, off_t >>> & ranges)

Parameters

Returns

A batch of op handles.

Header

#include <boost/afio/v2/afio.hpp>

Complexity

Amortised O(N) to dispatch. Amortised O(N/threadpool) to complete if deallocation is constant time.

Exception Model

Propagates exceptions of any input preconditions with an errored state at the point of dispatch, and throws a std::invalid_argument if any inputs have values which could not possibly be correct. Once a batch of input ops has been verified at the point of entry as not errored, you are guaranteed that the batch is atomically scheduled as a whole, unless a failure to allocate memory occurs.

Example

// Create a dispatcher
auto dispatcher = make_dispatcher().get();
// Schedule opening the log file for hole punching
auto logfilez(dispatcher->file(path_req("testdir/log", file_flags::create | file_flags::read_write)));
// Schedule opening the log file for atomic appending of log entries
auto logfilea(dispatcher->file(path_req("testdir/log", file_flags::create | file_flags::write | file_flags::append)));
// Retrieve any errors which occurred
logfilez.get();
logfilea.get();
// Initialise a random number generator
ranctx ctx;
raninit(&ctx, (u4) n);
while(!done)
{
  // Each log entry is 32 bytes in length
  union
  {
    char bytes[32];
    struct
    {
      uint64 id;      // The id of the writer
      uint64 r;       // A random number
      uint64 h1, h2;  // A hash of the previous two items
    };
  } buffer;
  buffer.id = n;
  buffer.r = ranval(&ctx);
  buffer.h1 = buffer.h2 = 1;
  SpookyHash::Hash128(buffer.bytes, 16, &buffer.h1, &buffer.h2);
  // Atomically append the log entry to the log file and wait for it
  // to complete, then fetch the new size of the log file.
  stat_t s = dispatcher->write(make_io_req(logfilea, buffer.bytes, 32, 0))->lstat();
  if(s.st_allocated > 8192 || s.st_size > 8192)
  {
    // Allocated space exceeds 8Kb. The actual file size reported may be
    // many times larger if the filing system supports hole punching.

    // Get the list of allocated extents
    std::vector<std::pair<off_t, off_t>> extents = dispatcher->extents(logfilez).get();
    // Filing system may not yet have allocated any storage ...
    if(!extents.empty())
    {
      if(extents.back().second > 1024)
        extents.back().second -= 1024;
      else
        extents.resize(extents.size() - 1);
      if(!extents.empty())
      {
        dispatcher->zero(logfilez, extents).get();
      }
    }
    else
      std::cout << "NOTE: extents() returns empty despite " << s.st_allocated << " bytes allocated (possibly delayed allocation)" << std::endl;
  }
}