Pimpl object of Engine. More...

Detailed Description

Pimpl object of Engine.

A private pimpl object for Engine. Do not include this header from a client program unless you know what you are doing.

Definition at line 52 of file engine_pimpl.hpp.

#include <engine_pimpl.hpp>

Inheritance diagram for foedus::EnginePimpl:

[legend]

Collaboration diagram for foedus::EnginePimpl:

[legend]

Classes
struct	ModulePtr
	pair of module pointer and its type. More...

Public Member Functions
	EnginePimpl ()=delete

	EnginePimpl (Engine *engine, const EngineOptions &options)

	EnginePimpl (Engine *engine, EngineType type, soc::Upid master_upid, Eid master_eid, soc::SocId soc_id)

bool	is_master () const

ErrorStack	initialize_once () override

ErrorStack	initialize_modules ()

ErrorStack	uninitialize_once () override

ErrorStack	check_valid_options ()

void	on_module_initialized (ModuleType module)
	Called whenever each module has completed its initialization. More...

void	on_module_uninitialized (ModuleType module)
	Called whenever each module has completed its uninitialization. More...

std::vector< ModulePtr >	get_modules ()
	Returns in initialization order. More...

std::string	describe_short () const

Public Member Functions inherited from foedus::DefaultInitializable
	DefaultInitializable ()

virtual	~DefaultInitializable ()

	DefaultInitializable (const DefaultInitializable &)=delete

DefaultInitializable &	operator= (const DefaultInitializable &)=delete

ErrorStack	initialize () override final
	Typical implementation of Initializable::initialize() that provides initialize-once semantics. More...

ErrorStack	uninitialize () override final
	Typical implementation of Initializable::uninitialize() that provides uninitialize-once semantics. More...

bool	is_initialized () const override final
	Returns whether the object has been already initialized or not. More...

Public Member Functions inherited from foedus::Initializable
virtual	~Initializable ()

Public Attributes
EngineOptions	options_
	Options given at boot time. More...

Engine *const	engine_
	Pointer to the enclosing object. More...

const EngineType	type_

const soc::Upid	master_upid_

const Eid	master_eid_

const soc::SocId	soc_id_

soc::SocManager	soc_manager_
	SOC manager. More...

debugging::DebuggingSupports	debug_
	Debugging supports. More...

proc::ProcManager	proc_manager_

memory::EngineMemory	memory_manager_

savepoint::SavepointManager	savepoint_manager_

thread::ThreadPool	thread_pool_

log::LogManager	log_manager_

snapshot::SnapshotManager	snapshot_manager_

cache::CacheManager	cache_manager_

storage::StorageManager	storage_manager_

xct::XctManager	xct_manager_

restart::RestartManager	restart_manager_

Constructor & Destructor Documentation

foedus::EnginePimpl::EnginePimpl ( )

delete

foedus::EnginePimpl::EnginePimpl	(	Engine *	engine,
		const EngineOptions &	options
	)

Definition at line 40 of file engine_pimpl.cpp.

                                                                      :
   options_(options),
   engine_(engine),
   type_(kMaster),
   master_upid_(::getpid()),
   // simply the pointer value of Engine object as identifier.
   master_eid_(reinterpret_cast<uintptr_t>(reinterpret_cast<void*>(engine))),
   soc_id_(0),
   // although we give a pointer to engine, these objects must not access it yet.
   // even the Engine object has not set the pimpl pointer.
   soc_manager_(engine),
   debug_(engine),
   proc_manager_(engine),
   memory_manager_(engine),
   savepoint_manager_(engine),
   thread_pool_(engine),
   log_manager_(engine),
   snapshot_manager_(engine),
   cache_manager_(engine),
   storage_manager_(engine),
   xct_manager_(engine),
   restart_manager_(engine) {
 }

foedus::EnginePimpl::EnginePimpl	(	Engine *	engine,
		EngineType	type,
		soc::Upid	master_upid,
		Eid	master_eid,
		soc::SocId	soc_id
	)

Definition at line 63 of file engine_pimpl.cpp.

                    :
   engine_(engine),
   type_(type),
   master_upid_(master_upid),
   master_eid_(master_eid),
   soc_id_(soc_id),
   soc_manager_(engine),
   debug_(engine),
   proc_manager_(engine),
   memory_manager_(engine),
   savepoint_manager_(engine),
   thread_pool_(engine),
   log_manager_(engine),
   snapshot_manager_(engine),
   cache_manager_(engine),
   storage_manager_(engine),
   xct_manager_(engine),
   restart_manager_(engine) {
 }

Member Function Documentation

ErrorStack foedus::EnginePimpl::check_valid_options ( )

Definition at line 213 of file engine_pimpl.cpp.

References CHECK_ERROR, foedus::kRetOk, options_, and foedus::EngineOptions::prescreen().

Referenced by initialize_once().

                                             {
   CHECK_ERROR(options_.prescreen(&std::cerr));
   CHECK_ERROR(check_minimal_pool_size());
   CHECK_ERROR(check_transparent_hugepage_setting());
   return kRetOk;
 }

Here is the call graph for this function:

Here is the caller graph for this function:

std::string foedus::EnginePimpl::describe_short ( ) const

Definition at line 88 of file engine_pimpl.cpp.

References foedus::kMaster, soc_id_, and type_.

Referenced by foedus::Engine::describe_short(), initialize_once(), and uninitialize_once().

                                             {
   if (type_ == kMaster) {
     return "MASTER";
   }
   std::string ret("CHILD-");
   return ret + std::to_string(soc_id_);
 }

Here is the caller graph for this function:

std::vector< ModulePtr > foedus::EnginePimpl::get_modules ( )

inline

Returns in initialization order.

Definition at line 127 of file engine_pimpl.hpp.

References foedus::kCache, foedus::kDebug, foedus::kLog, foedus::kMemory, foedus::kProc, foedus::kRestart, foedus::kSavepoint, foedus::kSnapshot, foedus::kStorage, foedus::kThread, and foedus::kXct.

Referenced by initialize_modules(), and uninitialize_once().

                                        {
     std::vector< ModulePtr > modules;
     // modules.push_back(ModulePtr(&soc_manager_, kSoc));  SOC Manager is specially inited/uninited
     modules.push_back(ModulePtr(&debug_, kDebug));
     modules.push_back(ModulePtr(&proc_manager_, kProc));
     modules.push_back(ModulePtr(&memory_manager_, kMemory));
     modules.push_back(ModulePtr(&savepoint_manager_, kSavepoint));
     modules.push_back(ModulePtr(&thread_pool_, kThread));
     modules.push_back(ModulePtr(&log_manager_, kLog));
     modules.push_back(ModulePtr(&snapshot_manager_, kSnapshot));
     modules.push_back(ModulePtr(&cache_manager_, kCache));
     modules.push_back(ModulePtr(&storage_manager_, kStorage));
     modules.push_back(ModulePtr(&xct_manager_, kXct));
     modules.push_back(ModulePtr(&restart_manager_, kRestart));
     return modules;
   }

Here is the caller graph for this function:

ErrorStack foedus::EnginePimpl::initialize_modules ( )

Definition at line 162 of file engine_pimpl.cpp.

References ASSERT_ND, CHECK_ERROR, get_modules(), foedus::soc::SocManager::is_initialized(), is_master(), foedus::kRetOk, on_module_initialized(), soc_manager_, foedus::soc::SocManager::wait_for_children_module(), and foedus::soc::SocManager::wait_for_master_module().

Referenced by initialize_once().

                                            {
   ASSERT_ND(soc_manager_.is_initialized());
   for (ModulePtr& module : get_modules()) {
     // During initialization, SOCs wait for master's initialization before their init.
     if (!is_master()) {
       CHECK_ERROR(soc_manager_.wait_for_master_module(true, module.type_));
     }
     CHECK_ERROR(module.ptr_->initialize());
     on_module_initialized(module.type_);
     // Then master waits for SOCs before moving on to next module.
     if (is_master()) {
       CHECK_ERROR(soc_manager_.wait_for_children_module(true, module.type_));
     }
   }
   return kRetOk;
 }

Here is the call graph for this function:

Here is the caller graph for this function:

ErrorStack foedus::EnginePimpl::initialize_once ( )

overridevirtual

Implements foedus::DefaultInitializable.

Definition at line 96 of file engine_pimpl.cpp.

References foedus::soc::SharedMemoryRepo::change_master_status(), CHECK_ERROR, check_valid_options(), foedus::soc::NodeMemoryAnchors::child_status_memory_, describe_short(), engine_, ERROR_STACK, foedus::soc::SharedMemoryRepo::get_node_memory_anchors(), foedus::Engine::get_options(), foedus::soc::SocManager::get_shared_memory_repo(), foedus::thread::ThreadOptions::group_count_, foedus::soc::SocManager::initialize(), initialize_modules(), foedus::ErrorStack::is_error(), is_master(), foedus::kErrorCodeSocChildInitFailed, foedus::soc::ChildEngineStatus::kFatalError, foedus::kRetOk, foedus::soc::MasterEngineStatus::kRunning, foedus::soc::ChildEngineStatus::kRunning, foedus::kSoc, foedus::assorted::memory_fence_acq_rel(), on_module_initialized(), foedus::soc::SocManager::report_engine_fatal_error(), soc_manager_, foedus::fs::status(), foedus::soc::ChildEngineStatus::status_code_, and foedus::EngineOptions::thread_.

                                         {
   if (is_master()) {
     CHECK_ERROR(check_valid_options());
   }
   // SOC manager is special. We must initialize it first.
   CHECK_ERROR(soc_manager_.initialize());
   on_module_initialized(kSoc);
   ErrorStack module_initialize_error = initialize_modules();
   if (module_initialize_error.is_error()) {
     LOG(ERROR) << "*******************************************************************************";
     LOG(ERROR) << "*** ERROR while module initailization in " << describe_short() << ". "
       << module_initialize_error << "";
     LOG(ERROR) << "*******************************************************************************";
     soc_manager_.report_engine_fatal_error();
     CHECK_ERROR(module_initialize_error);
   }
 
   // The following can assume SOC manager is already initialized
   if (is_master()) {
     soc::SharedMemoryRepo* repo = soc_manager_.get_shared_memory_repo();
     repo->change_master_status(soc::MasterEngineStatus::kRunning);
     // wait for children's kRunning status
     // TASK(Hideaki) should be a function in soc manager
     uint16_t soc_count = engine_->get_options().thread_.group_count_;
     while (true) {
       std::this_thread::sleep_for(std::chrono::milliseconds(5));
       assorted::memory_fence_acq_rel();
       bool error_happened = false;
       bool remaining = false;
       for (uint16_t node = 0; node < soc_count; ++node) {
         soc::ChildEngineStatus* status = repo->get_node_memory_anchors(node)->child_status_memory_;
         if (status->status_code_ == soc::ChildEngineStatus::kFatalError) {
           error_happened = true;
           break;
         }
         if (status->status_code_ == soc::ChildEngineStatus::kRunning) {
           continue;  // ok
         }
         remaining = true;
       }
 
       if (error_happened) {
         LOG(ERROR) << "[FOEDUS] ERROR! error while waiting child kRunning";
         soc_manager_.report_engine_fatal_error();
         return ERROR_STACK(kErrorCodeSocChildInitFailed);
       } else if (!remaining) {
         break;
       }
     }
   }
   LOG(INFO) << "================================================================================";
   LOG(INFO) << "================== FOEDUS ENGINE ("
     << describe_short() << ") INITIALIZATION DONE ===========";
   LOG(INFO) << "================================================================================";
 
   // In a few places, we check if we are running under valgrind and, if so, turn off
   // optimizations valgrind can't handle (eg hugepages).
   bool running_on_valgrind = RUNNING_ON_VALGRIND;
   if (running_on_valgrind) {
     LOG(INFO) << "=============== ATTENTION: VALGRIND MODE! ==================";
     LOG(INFO) << "This Engine is running under valgrind, which disables several optimizations";
     LOG(INFO) << "If you see this message while usual execution, something is wrong.";
     LOG(INFO) << "=============== ATTENTION: VALGRIND MODE! ==================";
   }
   return kRetOk;
 }

Here is the call graph for this function:

bool foedus::EnginePimpl::is_master ( ) const

inline

Definition at line 71 of file engine_pimpl.hpp.

References foedus::kMaster, and type_.

Referenced by initialize_modules(), initialize_once(), on_module_initialized(), on_module_uninitialized(), and uninitialize_once().

71 { return type_ == kMaster; }

foedus::EnginePimpl::type_

const EngineType type_

Definition: engine_pimpl.hpp:88

foedus::kMaster

The central instance that launches child engines on each NUMA node (SOC).

Definition: engine_type.hpp:41

Here is the caller graph for this function:

void foedus::EnginePimpl::on_module_initialized ( ModuleType module )

Called whenever each module has completed its initialization.

Definition at line 261 of file engine_pimpl.cpp.

References foedus::soc::MasterEngineStatus::change_init_atomic(), foedus::soc::ChildEngineStatus::change_init_atomic(), foedus::soc::NodeMemoryAnchors::child_status_memory_, foedus::soc::SharedMemoryRepo::get_global_memory_anchors(), foedus::soc::SharedMemoryRepo::get_node_memory_anchors(), foedus::soc::SocManager::get_shared_memory_repo(), is_master(), foedus::soc::GlobalMemoryAnchors::master_status_memory_, soc_id_, and soc_manager_.

Referenced by initialize_modules(), and initialize_once().

                                                          {
   soc::SharedMemoryRepo* repo = soc_manager_.get_shared_memory_repo();
   if (is_master()) {
     repo->get_global_memory_anchors()->master_status_memory_->change_init_atomic(module);
   } else {
     repo->get_node_memory_anchors(soc_id_)->child_status_memory_->change_init_atomic(module);
   }
 }

Here is the call graph for this function:

Here is the caller graph for this function:

void foedus::EnginePimpl::on_module_uninitialized ( ModuleType module )

Called whenever each module has completed its uninitialization.

Definition at line 270 of file engine_pimpl.cpp.

References foedus::soc::MasterEngineStatus::change_uninit_atomic(), foedus::soc::ChildEngineStatus::change_uninit_atomic(), foedus::soc::NodeMemoryAnchors::child_status_memory_, foedus::soc::SharedMemoryRepo::get_global_memory_anchors(), foedus::soc::SharedMemoryRepo::get_node_memory_anchors(), foedus::soc::SocManager::get_shared_memory_repo(), is_master(), foedus::soc::GlobalMemoryAnchors::master_status_memory_, soc_id_, and soc_manager_.

Referenced by uninitialize_once().

                                                            {
   soc::SharedMemoryRepo* repo = soc_manager_.get_shared_memory_repo();
   if (is_master()) {
     repo->get_global_memory_anchors()->master_status_memory_->change_uninit_atomic(module);
   } else {
     repo->get_node_memory_anchors(soc_id_)->child_status_memory_->change_uninit_atomic(module);
   }
 }

Here is the call graph for this function:

Here is the caller graph for this function:

ErrorStack foedus::EnginePimpl::uninitialize_once ( )

overridevirtual

Implements foedus::DefaultInitializable.

Definition at line 178 of file engine_pimpl.cpp.

References foedus::soc::SharedMemoryRepo::change_master_status(), describe_short(), foedus::ErrorStackBatch::emprace_back(), get_modules(), foedus::soc::SocManager::get_shared_memory_repo(), foedus::soc::SocManager::is_initialized(), is_master(), foedus::soc::MasterEngineStatus::kWaitingForChildTerminate, on_module_uninitialized(), soc_manager_, SUMMARIZE_ERROR_BATCH, foedus::soc::SocManager::uninitialize(), foedus::soc::SocManager::wait_for_children_module(), and foedus::soc::SocManager::wait_for_master_module().

                                           {
   LOG(INFO) << "================================================================================";
   LOG(INFO) << "=================== FOEDUS ENGINE ("
     << describe_short() << ") EXITTING...... ================";
   LOG(INFO) << "================================================================================";
   if (is_master() && soc_manager_.is_initialized()) {
     soc_manager_.get_shared_memory_repo()->change_master_status(
       soc::MasterEngineStatus::kWaitingForChildTerminate);
   }
   ErrorStackBatch batch;
   // uninit in reverse order of initialization
   auto modules = get_modules();
   std::reverse(modules.begin(), modules.end());
   for (ModulePtr& module : modules) {
     if (!module.ptr_->is_initialized()) {
       continue;
     }
     // During uninitialization, master waits for SOCs' uninitialization before its uninit.
     if (is_master()) {
       batch.emprace_back(soc_manager_.wait_for_children_module(false, module.type_));
     }
     batch.emprace_back(module.ptr_->uninitialize());
     on_module_uninitialized(module.type_);
     // Then SOCs wait for master before moving on to next module.
     if (!is_master()) {
       batch.emprace_back(soc_manager_.wait_for_master_module(false, module.type_));
     }
   }
 
   // SOC manager is special. We must uninitialize it at last.
   batch.emprace_back(soc_manager_.uninitialize());
   // after that, we can't even set status. shared memory has been detached.
   return SUMMARIZE_ERROR_BATCH(batch);
 }

Here is the call graph for this function:

Member Data Documentation

cache::CacheManager foedus::EnginePimpl::cache_manager_

Definition at line 121 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_cache_manager().

debugging::DebuggingSupports foedus::EnginePimpl::debug_

Debugging supports.

Attention: Because this module initializes/uninitializes basic debug logging support, EnginePimpl::initialize_once() must initialize it at the beginning, and EnginePimpl::uninitialize_once() must uninitialize it at the end. We should not use glog (except when we have to, in which case glog will give warnings) before and after that.

Definition at line 114 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_debug().

Engine* const foedus::EnginePimpl::engine_

Pointer to the enclosing object.

Few places would need it, but hold it in case.

Definition at line 86 of file engine_pimpl.hpp.

Referenced by initialize_once().

log::LogManager foedus::EnginePimpl::log_manager_

Definition at line 119 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_durable_global_epoch(), and foedus::Engine::get_log_manager().

const Eid foedus::EnginePimpl::master_eid_

Definition at line 90 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_master_eid().

const soc::Upid foedus::EnginePimpl::master_upid_

Definition at line 89 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_master_upid().

memory::EngineMemory foedus::EnginePimpl::memory_manager_

Definition at line 116 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_memory_manager().

EngineOptions foedus::EnginePimpl::options_

Options given at boot time.

Immutable once launched

Definition at line 83 of file engine_pimpl.hpp.

Referenced by check_valid_options(), foedus::Engine::get_nonconst_options(), foedus::Engine::get_options(), and foedus::Engine::get_soc_count().

proc::ProcManager foedus::EnginePimpl::proc_manager_

Definition at line 115 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_proc_manager().

restart::RestartManager foedus::EnginePimpl::restart_manager_

Definition at line 124 of file engine_pimpl.hpp.

savepoint::SavepointManager foedus::EnginePimpl::savepoint_manager_

Definition at line 117 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_earliest_epoch(), and foedus::Engine::get_savepoint_manager().

snapshot::SnapshotManager foedus::EnginePimpl::snapshot_manager_

Definition at line 120 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_snapshot_manager().

const soc::SocId foedus::EnginePimpl::soc_id_

Definition at line 91 of file engine_pimpl.hpp.

Referenced by describe_short(), foedus::Engine::get_soc_id(), on_module_initialized(), and on_module_uninitialized().

soc::SocManager foedus::EnginePimpl::soc_manager_

SOC manager.

This is a quite special module that launches child SOC engines. We have to initialize this module before everything else, even before debug_ (glog 0.3.3 has an issue across fork(), see https://code.google.com/p/google-glog/issues/detail?id=101 https://code.google.com/p/google-glog/issues/detail?id=82 ).

Definition at line 105 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_soc_manager(), initialize_modules(), initialize_once(), on_module_initialized(), on_module_uninitialized(), and uninitialize_once().

storage::StorageManager foedus::EnginePimpl::storage_manager_

Definition at line 122 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_storage_manager().

thread::ThreadPool foedus::EnginePimpl::thread_pool_

Definition at line 118 of file engine_pimpl.hpp.

Referenced by foedus::Engine::get_thread_pool().

const EngineType foedus::EnginePimpl::type_

Definition at line 88 of file engine_pimpl.hpp.

Referenced by describe_short(), foedus::Engine::get_type(), foedus::Engine::is_emulated_child(), foedus::Engine::is_forked_child(), foedus::Engine::is_local_spawned_child(), is_master(), foedus::Engine::is_master(), and foedus::Engine::is_remote_spawned_child().