foedus::fs::DirectIoFile Class Reference

Represents an I/O stream on one file without filesystem caching. More...

Detailed Description

Represents an I/O stream on one file without filesystem caching.

This class bypasses filesystem caching for disk I/O for three purposes.

• Performance, as DBMS has better knowledge of disk accesses than OS.
• Memory footprint, as we might end up doubled memory consumption.
• Durability, as write-cache would undermine the synchronous flush during commit.

This class is used for two kinds of files.

• Data file, or snapshot files.
• Log file

  Todo:

  Support Windows. MUCH later.

Definition at line 48 of file direct_io_file.hpp.

#include <direct_io_file.hpp>

Public Types
enum	Constants { kInvalidDescriptor = -1 }
	Constant values. More...
enum	SeekType { kDirectIoSeekSet = 0, kDirectIoSeekCur, kDirectIoSeekEnd }
	Analogue of SEEK_SET/SEEK_CUR/SEEK_END in POSIX. More...
typedef int	file_descriptor
	Represents low-level file descriptor. More...

Public Member Functions
	DirectIoFile (const Path &path, const DeviceEmulationOptions &emulation=DeviceEmulationOptions())
	Constructs this object without opening it yet. More...
	~DirectIoFile ()
	Automatically closes the file if it is opened. More...
	DirectIoFile ()=delete
	DirectIoFile (const DirectIoFile &)=delete
DirectIoFile &	operator= (const DirectIoFile &)=delete
ErrorCode	open (bool read, bool write, bool append, bool create)
	Tries to open the file for the specified volume. More...
bool	is_opened () const
	Whether the file is already and successfully opened. More...
bool	close ()
	Close the file if not yet closed. More...
ErrorCode	read (uint64_t desired_bytes, foedus::memory::AlignedMemory *buffer)
	Sequentially read the given amount of contents from the current position. More...
ErrorCode	read (uint64_t desired_bytes, const foedus::memory::AlignedMemorySlice &slice)
	Memory slice version. More...
ErrorCode	read_raw (uint64_t desired_bytes, void *buffer)
	A version that receives a raw pointer that has to be aligned (be careful to use this ver). More...
ErrorCode	write (uint64_t desired_bytes, const foedus::memory::AlignedMemory &buffer)
	Sequentially write the given amount of contents from the current position. More...
ErrorCode	write (uint64_t desired_bytes, const foedus::memory::AlignedMemorySlice &slice)
	Memory slice version. More...
ErrorCode	write_raw (uint64_t desired_bytes, const void *buffer)
	A version that receives a raw pointer that has to be aligned (be careful to use this ver). More...
ErrorCode	truncate (uint64_t new_length, bool sync=false)
	Discard the content of the file after the given offset. More...
ErrorCode	seek (uint64_t offset, SeekType seek_type)
	Sets the position of the next byte to be written/extracted from/to the stream. More...
ErrorCode	sync ()
	Analogues of POSIX fsync(). More...
Path	get_path () const
DeviceEmulationOptions	get_emulation () const
file_descriptor	get_descriptor () const
uint64_t	get_current_offset () const
bool	is_read () const
bool	is_write () const
std::string	to_string () const

Friends
std::ostream &	operator<< (std::ostream &o, const DirectIoFile &v)

Member Typedef Documentation

typedef int foedus::fs::DirectIoFile::file_descriptor

Represents low-level file descriptor.

Definition at line 51 of file direct_io_file.hpp.

Member Enumeration Documentation

enum foedus::fs::DirectIoFile::Constants

Constant values.

Enumerator
kInvalidDescriptor	POSIX open() semantics says -1 is invalid or not-yet-opened.

Definition at line 53 of file direct_io_file.hpp.

                 {
    kInvalidDescriptor = -1,
  };

enum foedus::fs::DirectIoFile::SeekType

Analogue of SEEK_SET/SEEK_CUR/SEEK_END in POSIX.

Enumerator
kDirectIoSeekSet	The offset is set to offset bytes.
kDirectIoSeekCur	The offset is set to its current location plus offset bytes.
kDirectIoSeekEnd	The offset is set to the size of the file plus offset bytes.

Definition at line 59 of file direct_io_file.hpp.

                {
    kDirectIoSeekSet = 0,
    kDirectIoSeekCur,
    kDirectIoSeekEnd,
  };

Constructor & Destructor Documentation

foedus::fs::DirectIoFile::DirectIoFile	(	const Path &	path,
		const DeviceEmulationOptions &	emulation = DeviceEmulationOptions()
	)

Constructs this object without opening it yet.

Parameters

[in]	path	Path of the file to manipulate
[in]	emulation	Optional argument to emulate slower devices.

Definition at line 44 of file direct_io_file.cpp.

  : path_(path), emulation_(emulation),
  descriptor_(kInvalidDescriptor), read_(false), write_(false), current_offset_(0) {
}

foedus::fs::DirectIoFile::~DirectIoFile

(

)

Automatically closes the file if it is opened.

Definition at line 51 of file direct_io_file.cpp.

References close().

                            {
  close();
}

Here is the call graph for this function:

foedus::fs::DirectIoFile::DirectIoFile ( )

delete

foedus::fs::DirectIoFile::DirectIoFile ( const DirectIoFile &  )

delete

Member Function Documentation

bool foedus::fs::DirectIoFile::close

(

)

Close the file if not yet closed.

Returns

Whether successfully closed. Even it fails, the file descriptor is already invalid. You cannot call this method again to close it appropriately.

Definition at line 129 of file direct_io_file.cpp.

References ASSERT_ND, kInvalidDescriptor, and foedus::assorted::os_error().

Referenced by foedus::snapshot::SnapshotWriter::close(), foedus::externalize::Externalizable::save_to_file(), foedus::log::MetaLogger::uninitialize_once(), foedus::log::Logger::uninitialize_once(), and ~DirectIoFile().

                         {
  if (descriptor_ != kInvalidDescriptor) {
    int ret = ::close(descriptor_);
    LOG(INFO) << "DirectIoFile::close(): closed. " << *this;
    if (ret != 0) {
      // Error at file close is nasty, we can't do much. We just report it in log.
      LOG(ERROR) << "DirectIoFile::close(): error:" << foedus::assorted::os_error()
        << " file=" << *this << ".";
      ASSERT_ND(false);  // crash only in debug mode
    }
    descriptor_ = kInvalidDescriptor;
    return ret == 0;
  }
  return false;
}

Here is the call graph for this function:

Here is the caller graph for this function:

uint64_t foedus::fs::DirectIoFile::get_current_offset ( ) const

inline

Definition at line 173 of file direct_io_file.hpp.

Referenced by foedus::log::MetaLogger::initialize_once(), foedus::log::Logger::initialize_once(), foedus::snapshot::SnapshotWriter::open(), foedus::fs::operator<<(), foedus::log::operator<<(), and foedus::snapshot::DumpFileSortedBuffer::wind().

{ return current_offset_; }

Here is the caller graph for this function:

file_descriptor foedus::fs::DirectIoFile::get_descriptor ( ) const

inline

Definition at line 172 of file direct_io_file.hpp.

Referenced by foedus::fs::operator<<().

{ return descriptor_; }

Here is the caller graph for this function:

DeviceEmulationOptions foedus::fs::DirectIoFile::get_emulation ( ) const

inline

Definition at line 171 of file direct_io_file.hpp.

{ return emulation_; }

Path foedus::fs::DirectIoFile::get_path ( ) const

inline

Definition at line 170 of file direct_io_file.hpp.

Referenced by foedus::snapshot::SnapshotWriter::close(), foedus::fs::operator<<(), and foedus::snapshot::DumpFileSortedBuffer::to_string().

{ return path_; }

Here is the caller graph for this function:

bool foedus::fs::DirectIoFile::is_opened ( ) const

inline

Whether the file is already and successfully opened.

Definition at line 95 of file direct_io_file.hpp.

References kInvalidDescriptor.

Referenced by read_raw(), sync(), truncate(), and write_raw().

{ return descriptor_ != kInvalidDescriptor; }

Here is the caller graph for this function:

bool foedus::fs::DirectIoFile::is_read ( ) const

inline

Definition at line 174 of file direct_io_file.hpp.

Referenced by foedus::fs::operator<<().

{ return read_; }

Here is the caller graph for this function:

bool foedus::fs::DirectIoFile::is_write ( ) const

inline

Definition at line 175 of file direct_io_file.hpp.

Referenced by foedus::fs::operator<<(), and sync().

{ return write_; }

Here is the caller graph for this function:

ErrorCode foedus::fs::DirectIoFile::open	(	bool	read,
		bool	write,
		bool	append,
		bool	create
	)

Tries to open the file for the specified volume.

Parameters

[in]	read	whether to allow read accesses on the opened file
[in]	write	whether to allow write accesses on the opened file
[in]	append	whether to set initial offset at the end of the file
[in]	create	whether to create the file. if already exists, does nothing.

Definition at line 55 of file direct_io_file.cpp.

References foedus::fs::Path::c_str(), foedus::fs::create_directories(), foedus::fs::DeviceEmulationOptions::disable_direct_io_, foedus::fs::exists(), foedus::fs::file_size(), foedus::kErrorCodeFsAlreadyOpened, foedus::kErrorCodeFsFailedToOpen, foedus::kErrorCodeFsMkdirFailed, foedus::kErrorCodeOk, kInvalidDescriptor, foedus::assorted::os_error(), foedus::fs::Path::parent_path(), read(), and write().

Referenced by foedus::log::MetaLogger::initialize_once(), foedus::log::Logger::initialize_once(), foedus::snapshot::SnapshotWriter::open(), and foedus::externalize::Externalizable::save_to_file().

                                                                             {
  if (descriptor_ != kInvalidDescriptor) {
      LOG(ERROR) << "DirectIoFile::open(): already opened. this=" << *this;
    return kErrorCodeFsAlreadyOpened;
  }
  Path folder(path_.parent_path());
  if (!exists(folder)) {
    if (!create_directories(folder, true)) {
      if (exists(folder)) {
        LOG(INFO) << "Interesting. other thread has created the folder:" << folder;
      } else {
        LOG(ERROR) << "DirectIoFile::open(): failed to create parent folder: "
          << folder << ". err=" << assorted::os_error();
        return kErrorCodeFsMkdirFailed;
      }
    }
  }

  LOG(INFO) << "DirectIoFile::open(): opening: " << path_ << "..  read =" << read << " write="
    << write << ", append=" << append << ", create=" << create;
  int oflags = O_LARGEFILE;
  if (!emulation_.disable_direct_io_) {
    oflags |= O_DIRECT;
  }
  if (read) {
    if (write) {
      oflags |= O_RDWR;
    } else {
      oflags |= O_RDONLY;
    }
  } else if (write) {
    // oflags |= O_WRONLY;
    oflags |= O_RDWR;
  }
  if (append) {
    oflags |= O_APPEND;
  }
  if (create) {
    oflags |= O_CREAT;
  }
  mode_t permissions = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH;
  descriptor_ = ::open(path_.c_str(), oflags, permissions);

  // tmpfs (such as /tmp, /dev/shm) refuses to receive O_DIRECT, returning EINVAL (22).
  // In that case, let's retry without O_DIRECT flag. MySQL does similar thing, too.
  if (descriptor_ == kInvalidDescriptor && (oflags & O_DIRECT) == O_DIRECT && errno == EINVAL) {
    descriptor_ = ::open(path_.c_str(), oflags ^ O_DIRECT, permissions);
    if (descriptor_ != kInvalidDescriptor) {
      // Okay, O_DIRECT was the cause. Just complain. go on.
      LOG(WARNING) << "DirectIoFile::open(): O_DIRECT flag for " << path_
        << " was rejected and automatically removed. This usually means you specified"
        << " tmpfs, such as /tmp, /dev/shm. Such non-durable devices should be used only"
        << " for testing and performance experiments."
        << " Related URL: http://www.gossamer-threads.com/lists/linux/kernel/720702";
    }
    // else the normal error flow below.
  }

  if (descriptor_ == kInvalidDescriptor) {
    LOG(ERROR) << "DirectIoFile::open(): failed to open: " << path_
      << ". err=" << assorted::os_error();
    return kErrorCodeFsFailedToOpen;
  } else {
    read_ = read;
    write_ = write;
    current_offset_ = 0;
    if (append) {
      current_offset_ = file_size(path_);
    }
    LOG(INFO) << "DirectIoFile::open(): successfully opened. " << *this;
    return kErrorCodeOk;
  }
}

Here is the call graph for this function:

Here is the caller graph for this function:

DirectIoFile& foedus::fs::DirectIoFile::operator= ( const DirectIoFile &  )

delete

ErrorCode foedus::fs::DirectIoFile::read	(	uint64_t	desired_bytes,
		foedus::memory::AlignedMemory *	buffer
	)

Sequentially read the given amount of contents from the current position.

Parameters

[in]	desired_bytes	Number of bytes to read. If we can't read this many bytes, we return errors.
[out]	buffer	Memory to copy into. As this is Direct-IO, it must be aligned.

Precondition

is_opened()

buffer.get_size() >= desired_bytes

(buffer->get_alignment() & 0xFFF) == 0 (4kb alignment)

Definition at line 144 of file direct_io_file.cpp.

Referenced by open(), read_raw(), and foedus::snapshot::DumpFileSortedBuffer::wind().

                                                                                 {
  return read(desired_bytes, memory::AlignedMemorySlice(buffer));
}

Here is the caller graph for this function:

ErrorCode foedus::fs::DirectIoFile::read	(	uint64_t	desired_bytes,
		const foedus::memory::AlignedMemorySlice &	slice
	)

Memory slice version.

Definition at line 147 of file direct_io_file.cpp.

References ASSERT_ND, foedus::memory::AlignedMemorySlice::count_, foedus::memory::AlignedMemory::get_alignment(), foedus::memory::AlignedMemorySlice::get_block(), foedus::fs::is_odirect_aligned(), foedus::kErrorCodeFsBufferNotAligned, foedus::kErrorCodeFsBufferTooSmall, foedus::memory::AlignedMemorySlice::memory_, foedus::fs::DeviceEmulationOptions::null_device_, and read_raw().

                                                                                            {
  ASSERT_ND(!emulation_.null_device_);
  if (desired_bytes > buffer.count_) {
    LOG(ERROR) << "DirectIoFile::read(): too small buffer is given. desired_bytes="
      << desired_bytes << ", buffer=" << buffer;
    return kErrorCodeFsBufferTooSmall;
  } else if (!is_odirect_aligned(buffer.memory_->get_alignment())
      || !is_odirect_aligned(buffer.get_block())
      || !is_odirect_aligned(desired_bytes)) {
    LOG(ERROR) << "DirectIoFile::read(): non-aligned input is given. buffer=" << buffer
      << ", desired_bytes=" << desired_bytes;
    return kErrorCodeFsBufferNotAligned;
  }
  return read_raw(desired_bytes, buffer.get_block());
}

Here is the call graph for this function:

ErrorCode foedus::fs::DirectIoFile::read_raw	(	uint64_t	desired_bytes,
		void *	buffer
	)

A version that receives a raw pointer that has to be aligned (be careful to use this ver).

Definition at line 162 of file direct_io_file.cpp.

References ASSERT_ND, foedus::fs::DeviceEmulationOptions::disable_direct_io_, foedus::fs::DeviceEmulationOptions::emulated_read_kb_cycles_, foedus::fs::is_odirect_aligned(), is_opened(), foedus::kErrorCodeFsExcessRead, foedus::kErrorCodeFsNotOpened, foedus::kErrorCodeFsResultNotAligned, foedus::kErrorCodeFsTooShortRead, foedus::kErrorCodeOk, foedus::fs::DeviceEmulationOptions::null_device_, foedus::assorted::os_error(), read(), and foedus::debugging::wait_rdtsc_cycles().

Referenced by read(), foedus::cache::SnapshotFileSet::read_page(), and foedus::cache::SnapshotFileSet::read_pages().

                                                                      {
  ASSERT_ND(!emulation_.null_device_);
  if (!is_opened()) {
    LOG(ERROR) << "File not opened yet, or closed. this=" << *this;
    return kErrorCodeFsNotOpened;
  } else if (desired_bytes == 0) {
    return kErrorCodeOk;
  }

  // underlying POSIX filesystem might split the read for severel reasons. so, while loop.
  uint64_t total_read = 0;
  uint64_t remaining = desired_bytes;
  while (remaining > 0) {
    char* position = reinterpret_cast<char*>(buffer) + total_read;
    ASSERT_ND(is_odirect_aligned(position));
    ssize_t read_bytes = ::read(descriptor_, position, remaining);
    if (read_bytes <= 0) {
      // zero means end of file (unexpected). negative value means error.
      LOG(ERROR) << "DirectIoFile::read(): error. this=" << *this
        << ", total_read=" << total_read << ", desired_bytes=" << desired_bytes
        << ", remaining=" << remaining << ", read_bytes=" << read_bytes
        << ", err=" << assorted::os_error();
      return kErrorCodeFsTooShortRead;
    }

    if (static_cast<uint64_t>(read_bytes) > remaining) {
      LOG(ERROR) << "DirectIoFile::read(): wtf? this=" << *this
        << ", total_read=" << total_read << ", desired_bytes=" << desired_bytes
        << ", remaining=" << remaining << ", read_bytes=" << read_bytes
        << ", err=" << assorted::os_error();
      return kErrorCodeFsExcessRead;
    } else if (!emulation_.disable_direct_io_ && !is_odirect_aligned(read_bytes)) {
      LOG(FATAL) << "DirectIoFile::read(): wtf2? this=" << *this
        << ", total_read=" << total_read << ", desired_bytes=" << desired_bytes
        << ", remaining=" << remaining << ", read_bytes=" << read_bytes
        << ", err=" << assorted::os_error();
      return kErrorCodeFsResultNotAligned;
    }

    total_read += read_bytes;
    remaining -= read_bytes;
    current_offset_ += read_bytes;
    if (remaining > 0) {
      LOG(INFO) << "Interesting. POSIX read() didn't complete the reads in one call."
        << " total_read=" << total_read << ", desired_bytes=" << desired_bytes
        << ", remaining=" << remaining;
    }
  }
  if (emulation_.emulated_read_kb_cycles_ > 0) {
    debugging::wait_rdtsc_cycles(emulation_.emulated_read_kb_cycles_ * (desired_bytes >> 10));
  }
  return kErrorCodeOk;
}

Here is the call graph for this function:

Here is the caller graph for this function:

ErrorCode foedus::fs::DirectIoFile::seek	(	uint64_t	offset,
		SeekType	seek_type
	)

Sets the position of the next byte to be written/extracted from/to the stream.

Precondition

is_opened()

Definition at line 326 of file direct_io_file.cpp.

References foedus::fs::DeviceEmulationOptions::emulated_seek_latency_cycles_, foedus::fs::is_odirect_aligned(), kDirectIoSeekCur, kDirectIoSeekEnd, kDirectIoSeekSet, foedus::kErrorCodeFsBadSeekInput, foedus::kErrorCodeFsBufferNotAligned, foedus::kErrorCodeFsSeekFailed, foedus::kErrorCodeOk, foedus::fs::DeviceEmulationOptions::null_device_, foedus::assorted::os_error(), and foedus::debugging::wait_rdtsc_cycles().

Referenced by foedus::cache::SnapshotFileSet::read_page(), and foedus::cache::SnapshotFileSet::read_pages().

                                                                 {
  if (!is_odirect_aligned(offset)) {
    LOG(ERROR) << "DirectIoFile::seek(): non-aligned input is given. offset=" << offset;
    return kErrorCodeFsBufferNotAligned;
  }
  if (emulation_.null_device_) {
    return kErrorCodeOk;
  }
  __off_t ret;
  switch (seek_type) {
    case kDirectIoSeekSet:
      ret = ::lseek(descriptor_, offset, SEEK_SET);
      break;
    case kDirectIoSeekCur:
      ret = ::lseek(descriptor_, offset, SEEK_CUR);
      break;
    case kDirectIoSeekEnd:
      ret = ::lseek(descriptor_, offset, SEEK_END);
      break;
    default:
      LOG(ERROR) << "DirectIoFile::seek(): wtf?? seek_type=" << seek_type;
      return kErrorCodeFsBadSeekInput;
  }
  if (ret < 0) {
    LOG(ERROR) << "DirectIoFile::seek(): failed. this=" << *this << ",err=" << assorted::os_error();
    return kErrorCodeFsSeekFailed;
  }
  current_offset_ = ret;
  if (emulation_.emulated_seek_latency_cycles_ > 0) {
    debugging::wait_rdtsc_cycles(emulation_.emulated_seek_latency_cycles_);
  }
  return kErrorCodeOk;
}

Here is the call graph for this function:

Here is the caller graph for this function:

ErrorCode foedus::fs::DirectIoFile::sync

(

)

Analogues of POSIX fsync().

POSIX fsync()

Transfers ("flushes") all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor fd to the disk device (or other permanent storage device) so that all changed information can be retrieved even after the system crashed or was rebooted.

No fdatasync analogue

All of our data writes are appends. So, there is no case we are benefited by fdatasync. Hence, we have only fsync() analogue.

Precondition

is_opened()

is_write()

Definition at line 360 of file direct_io_file.cpp.

References foedus::fs::fsync(), is_opened(), is_write(), foedus::kErrorCodeFsNotOpened, foedus::kErrorCodeFsSyncFailed, foedus::kErrorCodeInvalidParameter, foedus::kErrorCodeOk, foedus::fs::DeviceEmulationOptions::null_device_, and foedus::assorted::os_error().

                              {
  if (!is_opened()) {
    LOG(ERROR) << "File not opened yet, or closed. this=" << *this;
    return kErrorCodeFsNotOpened;
  }
  if (!is_write()) {
    return kErrorCodeInvalidParameter;
  }

  if (emulation_.null_device_) {
    return kErrorCodeOk;
  }

  int ret = ::fsync(descriptor_);
  if (ret != 0) {
    LOG(ERROR) << "DirectIoFile::sync(): fsync failed. this=" << *this
      << ", err=" << assorted::os_error();
    return kErrorCodeFsSyncFailed;
  }

  return kErrorCodeOk;
}

Here is the call graph for this function:

std::string foedus::fs::DirectIoFile::to_string

(

)

const

Definition at line 383 of file direct_io_file.cpp.

                                        {
  std::stringstream stream;
  stream << *this;
  return stream.str();
}

ErrorCode foedus::fs::DirectIoFile::truncate	(	uint64_t	new_length,
		bool	sync = false
	)

Discard the content of the file after the given offset.

Parameters

[in]	new_length	The size of this file would become this value.
[in]	sync	Whether to call fsync on the file and its parent folder after the truncation.

After a successful call of this method, the cursor would be at new_length, too. This method is used only when we restart the engine in order to discard non-durable parts of log files (regions after durable watermark).

Precondition

is_opened()

Definition at line 296 of file direct_io_file.cpp.

References foedus::fs::fsync(), foedus::fs::is_odirect_aligned(), is_opened(), foedus::kErrorCodeFsBufferNotAligned, foedus::kErrorCodeFsNotOpened, foedus::kErrorCodeFsTruncateFailed, foedus::kErrorCodeOk, foedus::fs::DeviceEmulationOptions::null_device_, and foedus::assorted::os_error().

Referenced by foedus::log::Logger::initialize_once().

                                                                {
  if (!is_odirect_aligned(new_length)) {
    LOG(ERROR) << "DirectIoFile::truncate(): non-aligned input is given. "
      << " new_length=" << new_length;
    return kErrorCodeFsBufferNotAligned;
  }
  LOG(INFO) << "DirectIoFile::truncate(): truncating " << *this << " to " << new_length
    << " bytes..";
  if (!is_opened()) {
    return kErrorCodeFsNotOpened;
  }

  if (emulation_.null_device_) {
    current_offset_ = new_length;
    return kErrorCodeOk;
  }

  if (::ftruncate(descriptor_, new_length) != 0) {
    LOG(ERROR) << "DirectIoFile::truncate(): failed. this=" << *this
      << " err=" << assorted::os_error();
    return kErrorCodeFsTruncateFailed;
  }
  current_offset_ = new_length;
  if (sync) {
    LOG(INFO) << "DirectIoFile::truncate(): also fsync..";
    foedus::fs::fsync(path_, true);
  }
  return kErrorCodeOk;
}

Here is the call graph for this function:

Here is the caller graph for this function:

ErrorCode foedus::fs::DirectIoFile::write	(	uint64_t	desired_bytes,
		const foedus::memory::AlignedMemory &	buffer
	)

Sequentially write the given amount of contents from the current position.

Parameters

[in]	desired_bytes	Number of bytes to write. If we can't write this many bytes, we return errors.
[in]	buffer	Memory to read from. As this is Direct-IO, it must be aligned.

Precondition

is_opened()

buffer.get_size() >= desired_bytes

(buffer->get_alignment() & 0xFFF) == 0 (4kb alignment)

Definition at line 216 of file direct_io_file.cpp.

Referenced by foedus::snapshot::SnapshotWriter::open(), open(), foedus::externalize::Externalizable::save_to_file(), and write_raw().

                                                                                        {
  return write(desired_bytes, memory::AlignedMemorySlice(
    const_cast<memory::AlignedMemory*>(&buffer)));
}

Here is the caller graph for this function:

ErrorCode foedus::fs::DirectIoFile::write	(	uint64_t	desired_bytes,
		const foedus::memory::AlignedMemorySlice &	slice
	)

Memory slice version.

Definition at line 221 of file direct_io_file.cpp.

References ASSERT_ND, foedus::memory::AlignedMemorySlice::count_, foedus::memory::AlignedMemory::get_alignment(), foedus::memory::AlignedMemorySlice::get_block(), foedus::fs::is_odirect_aligned(), foedus::memory::AlignedMemorySlice::is_valid(), foedus::kErrorCodeFsBufferNotAligned, foedus::kErrorCodeFsBufferTooSmall, foedus::memory::AlignedMemorySlice::memory_, and write_raw().

                                                                                             {
  ASSERT_ND(buffer.is_valid());
  if (desired_bytes > buffer.count_) {
    LOG(ERROR) << "DirectIoFile::write(): too small buffer is given. desired_bytes="
      << desired_bytes << ", buffer=" << buffer;
    return kErrorCodeFsBufferTooSmall;
  } else if (!is_odirect_aligned(buffer.memory_->get_alignment())
      || !is_odirect_aligned(buffer.get_block())
      || !is_odirect_aligned(desired_bytes)) {
    LOG(ERROR) << "DirectIoFile::write(): non-aligned input is given. buffer=" << buffer
      << ", desired_bytes=" << desired_bytes;
    return kErrorCodeFsBufferNotAligned;
  }
  return write_raw(desired_bytes, buffer.get_block());
}

Here is the call graph for this function:

ErrorCode foedus::fs::DirectIoFile::write_raw	(	uint64_t	desired_bytes,
		const void *	buffer
	)

A version that receives a raw pointer that has to be aligned (be careful to use this ver).

Definition at line 236 of file direct_io_file.cpp.

References ASSERT_ND, foedus::fs::DeviceEmulationOptions::disable_direct_io_, foedus::fs::DeviceEmulationOptions::emulated_write_kb_cycles_, foedus::fs::is_odirect_aligned(), is_opened(), foedus::kErrorCodeFsExcessWrite, foedus::kErrorCodeFsNotOpened, foedus::kErrorCodeFsResultNotAligned, foedus::kErrorCodeFsWriteFail, foedus::kErrorCodeOk, foedus::fs::DeviceEmulationOptions::null_device_, foedus::assorted::os_error(), foedus::debugging::wait_rdtsc_cycles(), and write().

Referenced by write().

                                                                             {
  if (!is_opened()) {
    LOG(ERROR) << "File not opened yet, or closed. this=" << *this;
    return kErrorCodeFsNotOpened;
  } else if (desired_bytes == 0) {
    return kErrorCodeOk;
  }

  if (emulation_.null_device_) {
    return kErrorCodeOk;
  }

  // underlying POSIX filesystem might split the write for severel reasons. so, while loop.
  VLOG(1) << "DirectIoFile::write(). desired_bytes=" << desired_bytes << ", buffer=" << buffer;
  uint64_t total_written = 0;
  uint64_t remaining = desired_bytes;
  while (remaining > 0) {
    const void* position = reinterpret_cast<const char*>(buffer) + total_written;
    VLOG(1) << "DirectIoFile::write(). position=" << position;
    ASSERT_ND(is_odirect_aligned(position));
    ssize_t written_bytes = ::write(descriptor_, position, remaining);
    if (written_bytes < 0) {
      // negative value means error.
      LOG(ERROR) << "DirectIoFile::write(): error. this=" << *this
        << ", total_written=" << total_written << ", desired_bytes=" << desired_bytes
        << ", remaining=" << remaining << ", written_bytes=" << written_bytes
        << ", err=" << assorted::os_error();
      // TASK(Hideaki) more error codes depending on errno. but mostly it should be disk-full
      return kErrorCodeFsWriteFail;
    }

    if (static_cast<uint64_t>(written_bytes) > remaining) {
      LOG(ERROR) << "DirectIoFile::write(): wtf? this=" << *this
        << ", total_written=" << total_written << ", desired_bytes=" << desired_bytes
        << ", remaining=" << remaining << ", written_bytes=" << written_bytes
        << ", err=" << assorted::os_error();
      return kErrorCodeFsExcessWrite;
    } else if (!emulation_.disable_direct_io_ && !is_odirect_aligned(written_bytes)) {
      LOG(FATAL) << "DirectIoFile::write(): wtf2? this=" << *this
        << ", total_written=" << total_written << ", desired_bytes=" << desired_bytes
        << ", remaining=" << remaining << ", written_bytes=" << written_bytes
        << ", err=" << assorted::os_error();
      return kErrorCodeFsResultNotAligned;
    }

    total_written += written_bytes;
    remaining -= written_bytes;
    current_offset_ += written_bytes;
    if (remaining > 0) {
      LOG(INFO) << "Interesting. POSIX write() didn't complete the writes in one call."
        << " total_written=" << total_written << ", desired_bytes=" << desired_bytes
        << ", remaining=" << remaining;
    }
  }
  if (emulation_.emulated_write_kb_cycles_ > 0) {
    debugging::wait_rdtsc_cycles(emulation_.emulated_write_kb_cycles_ * (desired_bytes >> 10));
  }
  return kErrorCodeOk;
}

Here is the call graph for this function:

Here is the caller graph for this function:

Friends And Related Function Documentation

std::ostream& operator<< ( std::ostream &  o, const DirectIoFile &  v  )

friend

Definition at line 388 of file direct_io_file.cpp.

                                                             {
  o << "<DirectIoFile>"
    << "<path>" << v.get_path() << "</path>"
    << "<descriptor>" << v.get_descriptor() << "</descriptor>"
    << "<read>" << v.is_read() << "</read>"
    << "<write>" << v.is_write() << "</write>"
    << "<current_offset>" << v.get_current_offset() << "</current_offset>"
    << "</DirectIoFile>";
  return o;
}

---

The documentation for this class was generated from the following files:

• /home/shino/foedus_code/foedus-core/include/foedus/fs/direct_io_file.hpp
• /home/shino/foedus_code/foedus-core/src/foedus/fs/direct_io_file.cpp