Avoiding the overhead of copying

Content

The presented solution is still less than optimal. While we construct new generators only when needed, we copy them unnecessarily. Have a look at the get method which lends an already present object:

T rg = unused.front();
unused.pop_front();
return rg;

In this case, the object from the list gets unnecessarily copied into the local variable. We have the same problem when an object is returned to the pool:

void free(T engine) {
   std::lock_guard<std::mutex> lock(mutex);
   unused.push_back(engine);
}

In this case the returned generator is possibly copied first when passed as parameter and then subsequently when push_back is invoked.

The simplest solution to this problem is the use of pointers as the copying of pointers is cheap. However, whenever we use pointers we have the open question who is responsible for releasing the associated memory. In C++ we do not decide this ad-hoc but have always a class that follows the RAII principle whose destructor takes care of that. In principle, it would be possible to use so-called smart pointers like std::shared_ptr<T> to solve this problem. However, they do not come for free as you would get an extra indirection whenever a value is to be retrieved from the pseudo-random generator.

Move semantics

C++11 provides an alternative approach by supporting move semantics. The principal idea is that the (non-trivial) contents of an object can be moved from one object to another at small cost. This is similar to a thread assignment where a joinable thread can be moved from a temporary object to an empty thread variable. As with threads move semantics can be quite useful for assignments where we have a temporary object on the right-hand side that holds voluminous dynamic data structures. In case of std::mt19937 the effect may be limited as this generator probably has no internal pointers in conjunction with dynamic data structures. But as our pool is to be written independently from a particular implementation, we should not ignore a possible copying overhead that could be avoided by using move semantics.

The idea is that we treat an object as if it is a temporary one by converting it in a so-called rvalue reference, i.e. a reference to an object where a move operation is permitted as it is no longer required afterwards, i.e. just the destructor has to work properly. In C++ we use && for rvalue references. Usually just temporary values can be passed to a rvalue reference. However, even non-temporary objects can be converted using std::move into an rvalue reference. Let us see how this can be done in the free method:

void free(T&& engine) {
   std::lock_guard<std::mutex> lock(mutex);
   unused.push_back(engine);
}

Now, we insist on an rvalue reference for the parameter to the free method, i.e. std::move has to be used when the engine is returned. This makes sense as the caller is not allowed to use the engine afterwards. Likewise, the std::deque class provides a push_back method with an rvalue reference.

Within the get method we can apply std::move to avoid copying:

T rg = std::move(unused.front());
unused.pop_front();

The front method returns an lvalue reference, i.e. T& but std::move converts this into an rvalue reference, i.e. T&&. Now, the compiler has the freedom to use the move constructor (i.e. T(T&& other)) to construct the rg object instead of using the copy constructor (i.e. T(T& other)). This possibly leaves the object in front of the list in an empty state which is subsequently discarded using the pop_front() method.

Exercise

To simplify the use of the adapted template class RandomEnginePool we need a RAII class RandomEngineGuard that

It should be possible to use this class as in following example:

template <
   template<typename> class MatrixA, typename T,
   typename POOL,
   Require< Ge<MatrixA<T>> > = true
>
void randomInit(MatrixA<T>& A, POOL& pool) {
   using EngineType = typename POOL::EngineType;
   RandomEngineGuard<EngineType> guard(pool);
   std::uniform_real_distribution<double> uniform(-100, 100);
   auto& engine = guard.get();

   for (auto [i, j, Aij]: A) {
      Aij = uniform(engine);
      (void) i; (void) j; // suppress gcc warning
   }
}