foedus::storage::StorageManagerPimpl Class Reference

Pimpl object of StorageManager. More...

Detailed Description

Pimpl object of StorageManager.

A private pimpl object for StorageManager. Do not include this header from a client program unless you know what you are doing.

Definition at line 73 of file storage_manager_pimpl.hpp.

#include <storage_manager_pimpl.hpp>

Inheritance diagram for foedus::storage::StorageManagerPimpl:

Collaboration diagram for foedus::storage::StorageManagerPimpl:

Public Member Functions
	StorageManagerPimpl ()=delete
	StorageManagerPimpl (Engine *engine)
ErrorStack	initialize_once () override
ErrorStack	initialize_read_latest_snapshot ()
ErrorStack	uninitialize_once () override
ErrorStack	reinitialize_for_recovered_snapshot ()
	Special method called only from recovery manager. More...
StorageId	issue_next_storage_id ()
StorageControlBlock *	get_storage (StorageId id)
StorageControlBlock *	get_storage (const StorageName &name)
bool	exists (const StorageName &name)
ErrorStack	drop_storage (StorageId id, Epoch *commit_epoch)
void	drop_storage_apply (StorageId id)
ErrorStack	create_storage (Metadata *metadata, Epoch *commit_epoch)
void	create_storage_apply (const Metadata &metadata)
template<typename STORAGE >
ErrorStack	create_storage_and_log (const Metadata *meta, Epoch *commit_epoch)
ErrorStack	hcc_reset_all_temperature_stat (StorageId storage_id)
	Resets all volatile pages' temperature stat to be zero in the specified storage. More...
xct::TrackMovedRecordResult	track_moved_record (StorageId storage_id, xct::RwLockableXctId *old_address, xct::WriteXctAccess *write)
ErrorStack	clone_all_storage_metadata (snapshot::SnapshotMetadata *metadata)
uint32_t	get_max_storages () const
Public Member Functions inherited from foedus::DefaultInitializable
	DefaultInitializable ()
virtual	~DefaultInitializable ()
	DefaultInitializable (const DefaultInitializable &)=delete
DefaultInitializable &	operator= (const DefaultInitializable &)=delete
ErrorStack	initialize () override final
	Typical implementation of Initializable::initialize() that provides initialize-once semantics. More...
ErrorStack	uninitialize () override final
	Typical implementation of Initializable::uninitialize() that provides uninitialize-once semantics. More...
bool	is_initialized () const override final
	Returns whether the object has been already initialized or not. More...
Public Member Functions inherited from foedus::Initializable
virtual	~Initializable ()

Public Attributes
Engine *const	engine_
StorageManagerControlBlock *	control_block_
StorageControlBlock *	storages_
	Storage instances (pimpl objects) are allocated in this shared memory. More...
storage::StorageId *	storage_name_sort_
	This shared memory stores the ID of storages sorted by their names. More...

Constructor & Destructor Documentation

foedus::storage::StorageManagerPimpl::StorageManagerPimpl ( )

delete

foedus::storage::StorageManagerPimpl::StorageManagerPimpl ( Engine *  engine )

inlineexplicit

Definition at line 76 of file storage_manager_pimpl.hpp.

: engine_(engine) {}

Member Function Documentation

ErrorStack foedus::storage::StorageManagerPimpl::clone_all_storage_metadata

(

snapshot::SnapshotMetadata *

metadata

)

Definition at line 470 of file storage_manager_pimpl.cpp.

References foedus::memory::AlignedMemory::alloc(), foedus::debugging::StopWatch::elapsed_ms(), foedus::memory::AlignedMemory::get_block(), foedus::memory::AlignedMemory::kNumaAllocOnnode, foedus::kRetOk, foedus::soc::GlobalMemoryAnchors::kStorageMemorySize, foedus::snapshot::SnapshotMetadata::largest_storage_id_, foedus::assorted::memory_fence_acq_rel(), foedus::debugging::StopWatch::stop(), foedus::snapshot::SnapshotMetadata::storage_control_blocks_, foedus::snapshot::SnapshotMetadata::storage_control_blocks_memory_, and storages_.

Referenced by foedus::storage::StorageManager::clone_all_storage_metadata().

                                      {
  debugging::StopWatch stop_watch;
  assorted::memory_fence_acq_rel();

  // not just the metadata, just copy the whole control block.
  // this is a single memcpy, which should be much more efficient.
  uint64_t memory_size
    = static_cast<uint64_t>(metadata->largest_storage_id_ + 1)
      * soc::GlobalMemoryAnchors::kStorageMemorySize;
  metadata->storage_control_blocks_memory_.alloc(
    memory_size,
    1 << 12,
    memory::AlignedMemory::kNumaAllocOnnode,
    0);
  metadata->storage_control_blocks_ = reinterpret_cast<storage::StorageControlBlock*>(
    metadata->storage_control_blocks_memory_.get_block());
  std::memcpy(metadata->storage_control_blocks_, storages_, memory_size);

  stop_watch.stop();
  LOG(INFO) << "Duplicated metadata of " << metadata->largest_storage_id_
    << " storages  in " << stop_watch.elapsed_ms() << " milliseconds";
  return kRetOk;
}

Here is the call graph for this function:

Here is the caller graph for this function:

ErrorStack foedus::storage::StorageManagerPimpl::create_storage	(	Metadata *	metadata,
		Epoch *	commit_epoch
	)

Definition at line 352 of file storage_manager_pimpl.cpp.

References ASSERT_ND, CHECK_ERROR, foedus::assorted::FixedString< MAXLEN, CHAR >::empty(), ERROR_STACK, exists(), get_max_storages(), get_storage(), foedus::storage::Metadata::id_, foedus::storage::StorageControlBlock::initialize(), foedus::INVALID_EPOCH, foedus::Epoch::is_valid(), issue_next_storage_id(), foedus::storage::kArrayStorage, foedus::kErrorCodeStrDuplicateStrname, foedus::kErrorCodeStrEmptyName, foedus::kErrorCodeStrTooManyStorages, foedus::storage::kHashStorage, foedus::storage::kMasstreeStorage, foedus::kRetOk, foedus::storage::kSequentialStorage, foedus::storage::StorageControlBlock::meta_, foedus::storage::Metadata::name_, storages_, foedus::storage::Metadata::type_, and foedus::assorted::FixedString< MAXLEN, CHAR >::zero_fill_remaining().

Referenced by foedus::storage::StorageManager::create_storage().

                                                                                      {
  *commit_epoch = INVALID_EPOCH;
  StorageId id = issue_next_storage_id();
  if (id >= get_max_storages()) {
    return ERROR_STACK(kErrorCodeStrTooManyStorages);
  }
  if (metadata->name_.empty()) {
    return ERROR_STACK(kErrorCodeStrEmptyName);
  }
  metadata->id_ = id;

  const StorageName& name = metadata->name_;
  if (exists(name)) {
    LOG(ERROR) << "This storage name already exists: " << name;
    return ERROR_STACK(kErrorCodeStrDuplicateStrname);
  }

  get_storage(id)->initialize();
  ASSERT_ND(!get_storage(id)->exists());
  storages_[id].meta_.type_ = metadata->type_;

  metadata->name_.zero_fill_remaining();  // make valgrind overload happy.
  ErrorStack result;
  if (metadata->type_ == kArrayStorage) {
    result = create_storage_and_log< array::ArrayStorage >(metadata, commit_epoch);
  } else if (metadata->type_ == kHashStorage) {
    result = create_storage_and_log< hash::HashStorage >(metadata, commit_epoch);
  } else if (metadata->type_ == kMasstreeStorage) {
    result = create_storage_and_log< masstree::MasstreeStorage >(metadata, commit_epoch);
  } else if (metadata->type_ == kSequentialStorage) {
    result = create_storage_and_log< sequential::SequentialStorage >(metadata, commit_epoch);
  } else {
    LOG(FATAL) << "WTF:" << metadata->type_;
  }
  CHECK_ERROR(result);

  ASSERT_ND(commit_epoch->is_valid());
  ASSERT_ND(get_storage(id)->exists());
  return kRetOk;
}

Here is the call graph for this function:

Here is the caller graph for this function:

template<typename STORAGE >

ErrorStack foedus::storage::StorageManagerPimpl::create_storage_and_log	(	const Metadata *	meta,
		Epoch *	commit_epoch
	)

Definition at line 321 of file storage_manager_pimpl.cpp.

References ASSERT_ND, CHECK_ERROR, foedus::log::MetaLogBuffer::commit(), engine_, foedus::storage::StorageControlBlock::exists(), foedus::Engine::get_log_manager(), foedus::log::LogManager::get_meta_buffer(), foedus::storage::Metadata::id_, foedus::kRetOk, foedus::storage::Metadata::name_, storages_, and foedus::storage::Metadata::type_.

                                                                                                {
  typedef typename STORAGE::ThisMetadata TheMetadata;
  typedef typename STORAGE::ThisCreateLogType TheLogType;

  StorageId id = meta->id_;
  StorageControlBlock* block = storages_ + id;
  STORAGE storage(engine_, block);
  const TheMetadata* casted_meta = reinterpret_cast< const TheMetadata *>(meta);
  ASSERT_ND(!block->exists());
  CHECK_ERROR(storage.create(*casted_meta));
  ASSERT_ND(block->exists());

  if (commit_epoch) {
    // if commit_epoch is null, it means "apply-only" mode in restart. do not log then
    char log_buffer[sizeof(TheLogType)];
    std::memset(log_buffer, 0, sizeof(log_buffer));
    TheLogType* create_log = reinterpret_cast<TheLogType*>(log_buffer);
    create_log->header_.storage_id_ = id;
    create_log->header_.log_type_code_ = log::get_log_code<TheLogType>();
    create_log->header_.log_length_ = sizeof(TheLogType);
    create_log->metadata_ = *casted_meta;
    ASSERT_ND(create_log->header_.storage_id_ == id);
    ASSERT_ND(create_log->metadata_.id_ == id);
    ASSERT_ND(create_log->metadata_.type_ == meta->type_);
    ASSERT_ND(create_log->metadata_.name_ == meta->name_);

    engine_->get_log_manager()->get_meta_buffer()->commit(create_log, commit_epoch);
  }
  return kRetOk;
}

Here is the call graph for this function:

void foedus::storage::StorageManagerPimpl::create_storage_apply

(

const Metadata &

metadata

)

Definition at line 393 of file storage_manager_pimpl.cpp.

References ASSERT_ND, control_block_, foedus::assorted::FixedString< MAXLEN, CHAR >::empty(), exists(), get_storage(), foedus::storage::Metadata::id_, foedus::storage::StorageControlBlock::initialize(), foedus::ErrorStack::is_error(), foedus::storage::kArrayStorage, foedus::storage::kHashStorage, foedus::storage::kMasstreeStorage, foedus::storage::kSequentialStorage, foedus::storage::StorageManagerControlBlock::largest_storage_id_, foedus::storage::StorageControlBlock::meta_, foedus::storage::Metadata::name_, storages_, and foedus::storage::Metadata::type_.

Referenced by foedus::storage::StorageManager::create_storage_apply().

                                                                       {
  // this method is called only while restart, so no race.
  ASSERT_ND(metadata.id_ > 0);
  ASSERT_ND(!metadata.name_.empty());
  StorageId id = metadata.id_;
  if (id > control_block_->largest_storage_id_) {
    control_block_->largest_storage_id_ = id;
  }

  ASSERT_ND(!exists(metadata.name_));

  get_storage(id)->initialize();
  ASSERT_ND(!get_storage(id)->exists());
  storages_[id].meta_.type_ = metadata.type_;
  ErrorStack result;
  if (metadata.type_ == kArrayStorage) {
    result = create_storage_and_log< array::ArrayStorage >(&metadata, nullptr);
  } else if (metadata.type_ == kHashStorage) {
    result = create_storage_and_log< hash::HashStorage >(&metadata, nullptr);
  } else if (metadata.type_ == kMasstreeStorage) {
    result = create_storage_and_log< masstree::MasstreeStorage >(&metadata, nullptr);
  } else if (metadata.type_ == kSequentialStorage) {
    result = create_storage_and_log< sequential::SequentialStorage >(&metadata, nullptr);
  } else {
    LOG(FATAL) << "WTF:" << metadata.type_;
  }
  if (result.is_error()) {
    LOG(FATAL) << "create_storage_apply() failed. " << result
      << " Failed to restart the engine";
  }

  ASSERT_ND(get_storage(id)->exists());
}

Here is the call graph for this function:

Here is the caller graph for this function:

ErrorStack foedus::storage::StorageManagerPimpl::drop_storage	(	StorageId	id,
		Epoch *	commit_epoch
	)

Definition at line 264 of file storage_manager_pimpl.cpp.

References ASSERT_ND, CHECK_ERROR, foedus::log::MetaLogBuffer::commit(), engine_, ERROR_STACK, foedus::storage::StorageControlBlock::exists(), foedus::Engine::get_log_manager(), foedus::log::LogManager::get_meta_buffer(), foedus::Epoch::is_valid(), foedus::storage::kArrayStorage, foedus::kErrorCodeStrAlreadyDropped, foedus::storage::kHashStorage, foedus::storage::kMarkedForDeath, foedus::storage::kMasstreeStorage, foedus::kRetOk, foedus::storage::kSequentialStorage, foedus::storage::StorageControlBlock::meta_, foedus::storage::Metadata::name_, foedus::storage::DropLogType::populate(), foedus::storage::StorageControlBlock::status_, storages_, foedus::storage::Metadata::type_, and foedus::storage::StorageControlBlock::uninitialize().

Referenced by foedus::storage::StorageManager::drop_storage().

                                                                              {
  StorageControlBlock* block = storages_ + id;
  if (!block->exists()) {
    LOG(ERROR) << "This storage ID does not exist or has been already dropped: " << id;
    return ERROR_STACK(kErrorCodeStrAlreadyDropped);
  }

  StorageName name = block->meta_.name_;
  LOG(INFO) << "Dropping storage " << id << "(" << name << ")";
  StorageType type = block->meta_.type_;
  if (type == kArrayStorage) {
    CHECK_ERROR(array::ArrayStorage(engine_, block).drop());
  } else if (type == kHashStorage) {
    CHECK_ERROR(hash::HashStorage(engine_, block).drop());
  } else if (type == kMasstreeStorage) {
    CHECK_ERROR(masstree::MasstreeStorage(engine_, block).drop());
  } else if (type == kSequentialStorage) {
    CHECK_ERROR(sequential::SequentialStorage(engine_, block).drop());
  } else {
    LOG(FATAL) << "WTF:" << type;
  }

  char log_buffer[1 << 12];
  std::memset(log_buffer, 0, sizeof(log_buffer));
  DropLogType* drop_log = reinterpret_cast<DropLogType*>(log_buffer);
  drop_log->populate(id);
  engine_->get_log_manager()->get_meta_buffer()->commit(drop_log, commit_epoch);

  ASSERT_ND(commit_epoch->is_valid());
  block->status_ = kMarkedForDeath;
  ASSERT_ND(!block->exists());
  block->uninitialize();
  LOG(INFO) << "Dropped storage " << id << "(" << name << ")";
  return kRetOk;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void foedus::storage::StorageManagerPimpl::drop_storage_apply

(

StorageId

id

)

Definition at line 300 of file storage_manager_pimpl.cpp.

References ASSERT_ND, COERCE_ERROR, engine_, foedus::storage::StorageControlBlock::exists(), foedus::storage::kArrayStorage, foedus::storage::kHashStorage, foedus::storage::kMarkedForDeath, foedus::storage::kMasstreeStorage, foedus::storage::kSequentialStorage, foedus::storage::StorageControlBlock::meta_, foedus::storage::StorageControlBlock::status_, storages_, foedus::storage::Metadata::type_, and foedus::storage::StorageControlBlock::uninitialize().

Referenced by foedus::storage::StorageManager::drop_storage_apply(), and uninitialize_once().

                                                         {
  StorageControlBlock* block = storages_ + id;
  ASSERT_ND(block->exists());
  StorageType type = block->meta_.type_;
  if (type == kArrayStorage) {
    COERCE_ERROR(array::ArrayStorage(engine_, block).drop());
  } else if (type == kHashStorage) {
    COERCE_ERROR(hash::HashStorage(engine_, block).drop());
  } else if (type == kMasstreeStorage) {
    COERCE_ERROR(masstree::MasstreeStorage(engine_, block).drop());
  } else if (type == kSequentialStorage) {
    COERCE_ERROR(sequential::SequentialStorage(engine_, block).drop());
  } else {
    LOG(FATAL) << "WTF:" << type;
  }
  block->status_ = kMarkedForDeath;
  ASSERT_ND(!block->exists());
  block->uninitialize();
}

Here is the call graph for this function:

Here is the caller graph for this function:

bool foedus::storage::StorageManagerPimpl::exists

(

const StorageName &

name

)

Definition at line 253 of file storage_manager_pimpl.cpp.

References control_block_, foedus::storage::StorageManagerControlBlock::largest_storage_id_, foedus::storage::StorageManagerControlBlock::mod_lock_, and storages_.

Referenced by create_storage(), create_storage_apply(), initialize_read_latest_snapshot(), and uninitialize_once().

                                                        {
  soc::SharedMutexScope guard(&control_block_->mod_lock_);
  // TASK(Hideaki) so far sequential search
  for (uint32_t i = 0; i <= control_block_->largest_storage_id_; ++i) {
    if (storages_[i].meta_.name_ == name) {
      return true;
    }
  }
  return false;
}

Here is the caller graph for this function:

uint32_t foedus::storage::StorageManagerPimpl::get_max_storages

(

)

const

Definition at line 65 of file storage_manager_pimpl.cpp.

References engine_, foedus::Engine::get_options(), foedus::storage::StorageOptions::max_storages_, and foedus::EngineOptions::storage_.

Referenced by create_storage().

                                                       {
  return engine_->get_options().storage_.max_storages_;
}

Here is the call graph for this function:

Here is the caller graph for this function:

StorageControlBlock* foedus::storage::StorageManagerPimpl::get_storage ( StorageId  id )

inline

Definition at line 93 of file storage_manager_pimpl.hpp.

References storages_.

Referenced by create_storage(), create_storage_apply(), foedus::storage::StorageManager::get_storage(), and initialize_read_latest_snapshot().

{ return &storages_[id]; }

Here is the caller graph for this function:

StorageControlBlock * foedus::storage::StorageManagerPimpl::get_storage

(

const StorageName &

name

)

Definition at line 242 of file storage_manager_pimpl.cpp.

References control_block_, foedus::storage::StorageManagerControlBlock::largest_storage_id_, foedus::storage::StorageManagerControlBlock::mod_lock_, and storages_.

                                                                             {
  soc::SharedMutexScope guard(&control_block_->mod_lock_);
  // TASK(Hideaki) so far sequential search
  for (uint32_t i = 0; i <= control_block_->largest_storage_id_; ++i) {
    if (storages_[i].meta_.name_ == name) {
      return &storages_[i];
    }
  }
  LOG(WARNING) << "Requested storage name '" << name << "' was not found";
  return &storages_[0];  // storage ID 0 is always not-initialized
}

ErrorStack foedus::storage::StorageManagerPimpl::hcc_reset_all_temperature_stat

(

StorageId

storage_id

)

Resets all volatile pages' temperature stat to be zero in the specified storage.

Used only in HCC-branch.

Definition at line 427 of file storage_manager_pimpl.cpp.

References ASSERT_ND, CHECK_ERROR, engine_, foedus::storage::StorageControlBlock::exists(), foedus::storage::kArrayStorage, foedus::storage::kHashStorage, foedus::storage::kMasstreeStorage, foedus::kRetOk, foedus::storage::StorageControlBlock::meta_, storages_, and foedus::storage::Metadata::type_.

                                                                                   {
  StorageControlBlock* block = storages_ + storage_id;
  ASSERT_ND(block->exists());
  StorageType type = block->meta_.type_;
  if (type == kMasstreeStorage) {
    CHECK_ERROR(masstree::MasstreeStorage(engine_, block).hcc_reset_all_temperature_stat());
  } else if (type == kHashStorage) {
    CHECK_ERROR(hash::HashStorage(engine_, block).hcc_reset_all_temperature_stat());
  } else if (type == kArrayStorage) {
    CHECK_ERROR(array::ArrayStorage(engine_, block).hcc_reset_all_temperature_stat());
  } else {
    // Seq storage doesn't need it.
    LOG(WARNING) << "This storage type doesn't need HCC counter reset. type=" << type;
  }
  return kRetOk;
}

Here is the call graph for this function:

ErrorStack foedus::storage::StorageManagerPimpl::initialize_once ( )

overridevirtual

Implements foedus::DefaultInitializable.

Definition at line 69 of file storage_manager_pimpl.cpp.

References CHECK_ERROR, control_block_, engine_, ERROR_STACK, foedus::soc::SharedMemoryRepo::get_global_memory_anchors(), foedus::Engine::get_log_manager(), foedus::soc::SocManager::get_shared_memory_repo(), foedus::Engine::get_soc_manager(), foedus::Engine::get_thread_pool(), foedus::storage::StorageManagerControlBlock::initialize(), initialize_read_latest_snapshot(), foedus::log::LogManager::is_initialized(), foedus::thread::ThreadPool::is_initialized(), foedus::Engine::is_master(), foedus::kErrorCodeDepedentModuleUnavailableInit, foedus::kRetOk, foedus::storage::StorageManagerControlBlock::largest_storage_id_, foedus::soc::GlobalMemoryAnchors::storage_manager_memory_, foedus::soc::GlobalMemoryAnchors::storage_memories_, storage_name_sort_, foedus::soc::GlobalMemoryAnchors::storage_name_sort_memory_, and storages_.

                                                {
  LOG(INFO) << "Initializing StorageManager..";
  if (!engine_->get_thread_pool()->is_initialized()
    || !engine_->get_log_manager()->is_initialized()) {
    return ERROR_STACK(kErrorCodeDepedentModuleUnavailableInit);
  }

  // attach shared memories
  soc::GlobalMemoryAnchors* anchors
    = engine_->get_soc_manager()->get_shared_memory_repo()->get_global_memory_anchors();
  control_block_ = anchors->storage_manager_memory_;
  storages_ = anchors->storage_memories_;
  storage_name_sort_ = anchors->storage_name_sort_memory_;

  if (engine_->is_master()) {
    // initialize the shared memory. only on master engine
    control_block_->initialize();
    control_block_->largest_storage_id_ = 0;

    // Then, initialize storages with latest snapshot
    CHECK_ERROR(initialize_read_latest_snapshot());
  }
  return kRetOk;
}

Here is the call graph for this function:

ErrorStack foedus::storage::StorageManagerPimpl::initialize_read_latest_snapshot

(

)

Definition at line 94 of file storage_manager_pimpl.cpp.

References ASSERT_ND, CHECK_ERROR, foedus::storage::VolatilePagePointer::clear(), control_block_, foedus::debugging::StopWatch::elapsed_ms(), foedus::assorted::FixedString< MAXLEN, CHAR >::empty(), engine_, ERROR_STACK, exists(), foedus::savepoint::SavepointManager::get_latest_snapshot_id(), foedus::Engine::get_savepoint_manager(), foedus::Engine::get_snapshot_manager(), get_storage(), foedus::storage::Metadata::id_, foedus::storage::StorageControlBlock::initialize(), foedus::Engine::is_master(), foedus::storage::kArrayStorage, foedus::kErrorCodeStrUnsupportedMetadata, foedus::storage::kExists, foedus::storage::kHashStorage, foedus::storage::kMasstreeStorage, foedus::storage::kNotExists, foedus::snapshot::kNullSnapshotId, foedus::kRetOk, foedus::storage::kSequentialStorage, foedus::storage::StorageManagerControlBlock::largest_storage_id_, foedus::snapshot::SnapshotMetadata::largest_storage_id_, foedus::storage::StorageControlBlock::meta_, foedus::storage::Metadata::name_, foedus::snapshot::SnapshotManager::read_snapshot_metadata(), foedus::storage::StorageControlBlock::root_page_pointer_, foedus::storage::Metadata::root_snapshot_page_id_, foedus::storage::DualPagePointer::snapshot_pointer_, foedus::storage::StorageControlBlock::status_, foedus::debugging::StopWatch::stop(), foedus::snapshot::SnapshotMetadata::storage_control_blocks_, storages_, foedus::storage::Metadata::type_, and foedus::storage::DualPagePointer::volatile_pointer_.

Referenced by initialize_once().

                                                                {
  ASSERT_ND(engine_->is_master());
  LOG(INFO) << "Initializing list of storages with latest snapshot file...";
  snapshot::SnapshotId snapshot_id = engine_->get_savepoint_manager()->get_latest_snapshot_id();
  if (snapshot_id == snapshot::kNullSnapshotId) {
    LOG(INFO) << "There was no previous snapshot. start from empty storages";
    return kRetOk;
  }

  snapshot::SnapshotMetadata metadata;
  CHECK_ERROR(engine_->get_snapshot_manager()->read_snapshot_metadata(snapshot_id, &metadata));
  LOG(INFO) << "Latest snapshot contains " << metadata.largest_storage_id_ << " storages";
  control_block_->largest_storage_id_ = metadata.largest_storage_id_;

  debugging::StopWatch stop_watch;
  uint32_t active_storages = 0;
  for (uint32_t id = 1; id <= control_block_->largest_storage_id_; ++id) {
    StorageControlBlock* block = storages_ + id;
    const StorageControlBlock& snapshot_block = metadata.storage_control_blocks_[id];
    if (snapshot_block.status_ != kExists) {
      VLOG(0) << "Storage-" << id << " is a dropped storage.";
      block->status_ = kNotExists;
    } else {
      VLOG(0) << "Storage-" << id << " exists in the snapshot.";
      ASSERT_ND(snapshot_block.meta_.id_ == id);
      ASSERT_ND(!snapshot_block.meta_.name_.empty());
      ASSERT_ND(!exists(snapshot_block.meta_.name_));
      block->initialize();
      block->root_page_pointer_.snapshot_pointer_ = snapshot_block.meta_.root_snapshot_page_id_;
      block->root_page_pointer_.volatile_pointer_.clear();

      block->meta_.type_ = snapshot_block.meta_.type_;
      switch (snapshot_block.meta_.type_) {
      case kArrayStorage:
        CHECK_ERROR(array::ArrayStorage(engine_, block).load(snapshot_block));
        break;
      case kHashStorage:
        CHECK_ERROR(hash::HashStorage(engine_, block).load(snapshot_block));
        break;
      case kMasstreeStorage:
        CHECK_ERROR(masstree::MasstreeStorage(engine_, block).load(snapshot_block));
        break;
      case kSequentialStorage:
        CHECK_ERROR(sequential::SequentialStorage(engine_, block).load(snapshot_block));
        break;
      default:
        return ERROR_STACK(kErrorCodeStrUnsupportedMetadata);
      }

      ASSERT_ND(get_storage(id)->exists());

      ++active_storages;
    }
  }
  stop_watch.stop();
  LOG(INFO) << "Found " << active_storages
    << " active storages  in " << stop_watch.elapsed_ms() << " milliseconds";
  return kRetOk;
}

Here is the call graph for this function:

Here is the caller graph for this function:

StorageId foedus::storage::StorageManagerPimpl::issue_next_storage_id

(

)

Definition at line 235 of file storage_manager_pimpl.cpp.

References control_block_, foedus::storage::StorageManagerControlBlock::largest_storage_id_, and foedus::storage::StorageManagerControlBlock::mod_lock_.

Referenced by create_storage(), and foedus::storage::StorageManager::issue_next_storage_id().

                                                     {
  soc::SharedMutexScope guard(&control_block_->mod_lock_);  // implies fence too
  ++control_block_->largest_storage_id_;
  LOG(INFO) << "Incremented largest_storage_id_: " << control_block_->largest_storage_id_;
  return control_block_->largest_storage_id_;
}

Here is the caller graph for this function:

ErrorStack foedus::storage::StorageManagerPimpl::reinitialize_for_recovered_snapshot

(

)

Special method called only from recovery manager.

1) initialize_once(), which calls initialize_read_latest_snapshot(). 2) if there are non-snapshotted and durable logs, recovery manager triggers snapshotting 3) then, reinitialize only root pages with this method.

Unlike usual snapshot, we can assume that there is no concurrent worker, so we can safely drop volatile root pages and re-create them with the new snapshot pages.

Definition at line 157 of file storage_manager_pimpl.cpp.

References ASSERT_ND, CHECK_ERROR, control_block_, foedus::debugging::StopWatch::elapsed_ms(), engine_, foedus::storage::StorageControlBlock::exists(), foedus::memory::EngineMemory::get_global_volatile_page_resolver(), foedus::storage::Page::get_header(), foedus::Engine::get_memory_manager(), foedus::storage::Page::get_snapshot_page_id(), foedus::storage::Page::get_volatile_page_id(), foedus::DefaultInitializable::initialize(), foedus::storage::VolatilePagePointer::is_null(), foedus::kRetOk, foedus::UninitializeGuard::kWarnIfUninitializeError, foedus::storage::StorageManagerControlBlock::largest_storage_id_, foedus::storage::PageHeader::page_id_, foedus::cache::SnapshotFileSet::read_page(), foedus::storage::StorageControlBlock::root_page_pointer_, foedus::storage::PageHeader::snapshot_, foedus::storage::DualPagePointer::snapshot_pointer_, foedus::debugging::StopWatch::stop(), foedus::storage::PageHeader::storage_id_, storages_, foedus::DefaultInitializable::uninitialize(), foedus::storage::DualPagePointer::volatile_pointer_, foedus::storage::VolatilePagePointer::word, and WRAP_ERROR_CODE.

Referenced by foedus::storage::StorageManager::reinitialize_for_recovered_snapshot().

                                                                    {
  LOG(INFO) << "Replacing existing, stale volatile root pages with the new root snapshot pages..";
  auto* memory_manager = engine_->get_memory_manager();
  const auto& resolver = memory_manager->get_global_volatile_page_resolver();
  debugging::StopWatch stop_watch;
  uint32_t refreshed_storages = 0;

  cache::SnapshotFileSet fileset(engine_);
  CHECK_ERROR(fileset.initialize());
  UninitializeGuard fileset_guard(&fileset, UninitializeGuard::kWarnIfUninitializeError);
  for (uint32_t id = 1; id <= control_block_->largest_storage_id_; ++id) {
    StorageControlBlock* block = storages_ + id;
    const SnapshotPagePointer snapshot_page_id = block->root_page_pointer_.snapshot_pointer_;
    const VolatilePagePointer volatile_page_id = block->root_page_pointer_.volatile_pointer_;
    if (!block->exists()) {
      continue;
    } else if (snapshot_page_id == 0) {
      continue;
    } else if (volatile_page_id.is_null()) {
      // some storage type allows null volatile root pages. In that case,
      // we don't have to do anything at this point. When the initial non-read-ony request
      // is made, the root volatile page will be automatically created.
      continue;
    }

    LOG(INFO) << "Re-initializing root of storage-" << id << " from the recovered snapshot."
      << "Volatile page ID=" << volatile_page_id << ", Snapshot page ID=" << snapshot_page_id;
    Page* volatile_page = resolver.resolve_offset(volatile_page_id);
    ASSERT_ND(!volatile_page->get_header().snapshot_);
    ASSERT_ND(volatile_page->get_header().storage_id_ == id);
    ASSERT_ND(volatile_page->get_volatile_page_id() == volatile_page_id);

    // Here, we assume that the initially-allocated volatile root page does NOT have
    // any child volatile page (it shouldn't!). Otherwise, the following overwrite
    // will cause leaked volatile pages.
    WRAP_ERROR_CODE(fileset.read_page(snapshot_page_id, volatile_page));
    ASSERT_ND(volatile_page->get_header().snapshot_);
    ASSERT_ND(volatile_page->get_header().storage_id_ == id);
    ASSERT_ND(volatile_page->get_snapshot_page_id() == snapshot_page_id);
    volatile_page->get_header().snapshot_ = false;
    volatile_page->get_header().page_id_ = volatile_page_id.word;
    ++refreshed_storages;
  }

  CHECK_ERROR(fileset.uninitialize());
  stop_watch.stop();
  LOG(INFO) << "Refreshed " << refreshed_storages
    << " storages with the recovered snapshot in " << stop_watch.elapsed_ms() << " milliseconds";
  return kRetOk;
}

Here is the call graph for this function:

Here is the caller graph for this function:

xct::TrackMovedRecordResult foedus::storage::StorageManagerPimpl::track_moved_record	(	StorageId	storage_id,
		xct::RwLockableXctId *	old_address,
		xct::WriteXctAccess *	write
	)

Definition at line 452 of file storage_manager_pimpl.cpp.

References ASSERT_ND, engine_, foedus::storage::StorageControlBlock::exists(), foedus::storage::kHashStorage, foedus::storage::kMasstreeStorage, foedus::storage::StorageControlBlock::meta_, storages_, foedus::storage::hash::HashStorage::track_moved_record(), foedus::storage::masstree::MasstreeStorage::track_moved_record(), and foedus::storage::Metadata::type_.

Referenced by foedus::storage::StorageManager::track_moved_record().

                                {
  // so far Masstree and Hash have tracking
  StorageControlBlock* block = storages_ + storage_id;
  ASSERT_ND(block->exists());
  StorageType type = block->meta_.type_;
  if (type == kMasstreeStorage) {
    return masstree::MasstreeStorage(engine_, block).track_moved_record(old_address, write_set);
  } else if (type == kHashStorage) {
    return hash::HashStorage(engine_, block).track_moved_record(old_address, write_set);
  } else {
    LOG(ERROR) << "Unexpected storage type for a moved-record. Bug? type=" << type;
    return xct::TrackMovedRecordResult();
  }
}

Here is the call graph for this function:

Here is the caller graph for this function:

ErrorStack foedus::storage::StorageManagerPimpl::uninitialize_once ( )

overridevirtual

Implements foedus::DefaultInitializable.

Definition at line 208 of file storage_manager_pimpl.cpp.

References ASSERT_ND, control_block_, drop_storage_apply(), foedus::ErrorStackBatch::emprace_back(), engine_, ERROR_STACK, exists(), foedus::Engine::get_log_manager(), foedus::Engine::get_thread_pool(), foedus::log::LogManager::is_initialized(), foedus::thread::ThreadPool::is_initialized(), foedus::Engine::is_master(), foedus::kErrorCodeDepedentModuleUnavailableUninit, foedus::storage::StorageManagerControlBlock::largest_storage_id_, storages_, SUMMARIZE_ERROR_BATCH, and foedus::storage::StorageManagerControlBlock::uninitialize().

                                                  {
  LOG(INFO) << "Uninitializing StorageManager..";
  ErrorStackBatch batch;
  if (!engine_->get_thread_pool()->is_initialized()
    || !engine_->get_log_manager()->is_initialized()) {
    batch.emprace_back(ERROR_STACK(kErrorCodeDepedentModuleUnavailableUninit));
  }
  if (engine_->is_master()) {
    // drop all existing storages just for releasing memories.
    // this is not a real drop, so we just invoke drop_apply
    uint32_t dropped = 0;
    for (storage::StorageId i = 1; i <= control_block_->largest_storage_id_; ++i) {
      if (storages_[i].exists()) {
        // TASK(Hideaki) we should have a separate method for this once we add "marked-for-death"
        // feature.
        drop_storage_apply(i);
        ++dropped;
        ASSERT_ND(!storages_[i].exists());
      }
    }
    LOG(INFO) << "Uninitialized " << dropped << " storages";

    control_block_->uninitialize();
  }
  return SUMMARIZE_ERROR_BATCH(batch);
}

Here is the call graph for this function:

Member Data Documentation

StorageManagerControlBlock* foedus::storage::StorageManagerPimpl::control_block_

Definition at line 120 of file storage_manager_pimpl.hpp.

Referenced by create_storage_apply(), exists(), foedus::storage::StorageManager::get_largest_storage_id(), get_storage(), initialize_once(), initialize_read_latest_snapshot(), issue_next_storage_id(), reinitialize_for_recovered_snapshot(), and uninitialize_once().

Engine* const foedus::storage::StorageManagerPimpl::engine_

Definition at line 118 of file storage_manager_pimpl.hpp.

Referenced by create_storage_and_log(), drop_storage(), drop_storage_apply(), foedus::storage::StorageManager::get_engine(), get_max_storages(), hcc_reset_all_temperature_stat(), initialize_once(), initialize_read_latest_snapshot(), reinitialize_for_recovered_snapshot(), track_moved_record(), and uninitialize_once().

storage::StorageId* foedus::storage::StorageManagerPimpl::storage_name_sort_

This shared memory stores the ID of storages sorted by their names.

Accessing this, either read or write, must take mod_lock_. This is why get_storage(string) is more expensive.

Definition at line 135 of file storage_manager_pimpl.hpp.

Referenced by initialize_once().

StorageControlBlock* foedus::storage::StorageManagerPimpl::storages_

Storage instances (pimpl objects) are allocated in this shared memory.

Each storage instance must be within 4kb. If the storage type requires more than 4kb, just grab a page from volatile page pools and point to it from the storage object. Remember that all pages in volatile page pools are shared.

Definition at line 128 of file storage_manager_pimpl.hpp.

Referenced by clone_all_storage_metadata(), create_storage(), create_storage_and_log(), create_storage_apply(), drop_storage(), drop_storage_apply(), exists(), get_storage(), hcc_reset_all_temperature_stat(), initialize_once(), initialize_read_latest_snapshot(), reinitialize_for_recovered_snapshot(), track_moved_record(), and uninitialize_once().

---

The documentation for this class was generated from the following files:

• /home/shino/foedus_code/foedus-core/include/foedus/storage/storage_manager_pimpl.hpp
• /home/shino/foedus_code/foedus-core/src/foedus/storage/storage_manager_pimpl.cpp