C++, Pimpl

Basics

You may come across the Pimpl template under other names: d-pointer, compiler firewall, or even Cheshire Cat template or opaque pointer.

In its main form, the template looks like this:

• In a class, we move all private members to a new declared type, such as the PrivateImpl class.
• We declare PrivateImpl in the header file of the main class.
• In the appropriate cpp file, we declare the class PrivateImpl and define it.
• Now if you change the private implementation, the client code will not be recompiled (because the interface hasn't changed).

So it might look like this (crude, old code style!):

class.h

class MyClassImpl;
class MyClass {
    // ...
    void Foo();
private:    
    MyClassImpl* m_pImpl; // Предупреждение!!! 
                          // raw-указатель! :)
};

class.cpp

class MyClassImpl
{
public:
    void DoStuff() { /*...*/ }
};

MyClass::MyClass () 
: m_pImpl(new MyClassImpl()) 
{ }

MyClass::~MyClass () { delete m_pImpl; }

void MyClass ::DoSth() {
    m_pImpl->DoSth();
}

Eh... ugly raw pointers!

So, in short: we pack everything that is private into this forward-declared class. We use only one member of our main class - the compiler can only work with a pointer without a full type declaration, since only the size of the pointer is needed. The declaration and implementation then happen in the .cpp file.

Of course, in modern C++ it is recommended to use unique_ptr rather than raw pointers.

The two obvious disadvantages of this approach are that we need to allocate memory to store the private section. And also the main class simply redirects the method call to the private implementation.

Okay...but that's it...right? Not so easy!

The above code might work, but we have to add a few bits to make it work in real life.

More code

We have to ask a few questions before we can write the full code:

• is your class copyable or only moveable?
• how to force const for methods in this private implementation?
• do you need a "backward" pointer so that the impl class can call/reference members of the main class?
• what should be included in this private implementation? everything that is private?

The first part, copy/relocatability, refers to the fact that with a simple raw pointer, we can only copy an object in a shallow way. Of course this happens every time you have a pointer in your class.

So, we must definitely implement a copy constructor (or remove it if we want to have only a relocatable type).

What about the problem with const ? Can you catch it in the main example?

If you declare a const method, you cannot change the members of the object. In other words, they become const . But this is a problem for our m_pImpl , which is a pointer. In a const method, that pointer will also become const, which means we can't assign a different value to it... but... we can happily call all methods of that base private class (not just constants)!

So we need a conversion/wrapper mechanism. Something like that:

const MyClassImpl* Pimpl() const { return m_pImpl; }
MyClassImpl* Pimpl() { return m_pImpl; }

And now, in all our main class methods, we should use this wrapper function, and not the pointer itself.

So far, I haven't mentioned this "reverse" pointer ("q-pointer" in Qt terminology). The answer is related to the last point - what should we inject into a private implementation - only private fields? Or maybe even private functions?

Basic code won't show these practical problems. But in a real application, a class can contain many methods and fields. I've seen examples where the whole private part (using methods) was passed into the pimpl class. However, sometimes the pimpl class needs to call the "real" method of the main class, so we need to provide that "backward" pointer. This can be done on construction, just pass a pointer to this .

Improved version

So here's an improved version of our sample code:

class.h

class MyClassImpl;
class MyClass
{
public:
    explicit MyClass();
    ~MyClass(); 

    // movable:
    MyClass(MyClass && rhs) noexcept;   
    MyClass& operator=(MyClass && rhs) noexcept;

    // and copyable
    MyClass(const MyClass& rhs);
    MyClass& operator=(const MyClass& rhs);

    void DoSth();
    void DoConst() const;

private:
    const MyClassImpl* Pimpl() const { return m_pImpl.get(); }
    MyClassImpl* Pimpl() { return m_pImpl.get(); }

    std::unique_ptr<MyClassImpl> m_pImpl;
};

class.cpp

class MyClassImpl
{
public:
    ~MyClassImpl() = default;

    void DoSth() { }
    void DoConst() const { }
};

MyClass::MyClass() : m_pImpl(new MyClassImpl()) 
{

}

MyClass::~MyClass() = default;
MyClass::MyClass(MyClass &&) noexcept = default;
MyClass& MyClass::operator=(MyClass &&) noexcept = default;

MyClass::MyClass(const MyClass& rhs)
    : m_pImpl(new MyClassImpl(*rhs.m_pImpl))
{}

MyClass& MyClass::operator=(const MyClass& rhs) {
    if (this != &rhs) 
        m_pImpl.reset(new MyClassImpl(*rhs.m_pImpl));

    return *this;
}

void MyClass::DoSth()
{
    Pimpl()->DoSth();
}

void MyClass::DoConst() const
{
    Pimpl()->DoConst();
}

Now a little better.

The above code uses

• unique_ptr - but see that the destructor for the main class must be defined in the cpp file. Otherwise, the compiler will complain about the missing type being removed...
• The class is movable and copyable because four methods are defined
• To be safe with const methods, all proxy methods of the main class use the Pimpl() method to get the correct pointer type.

Take a look at this block Pimp My Pimpl - Reloaded for more information about Pimpl.

You can play around with the full example here (it also has some nice stuff to learn)

cpp_pimpl2.h

#include <memory>

class MyClassImpl;
class MyClass
{
public:
    explicit MyClass();
    ~MyClass(); 

    // movable:
    MyClass(MyClass && rhs) noexcept;   
    MyClass& operator=(MyClass && rhs) noexcept;

    // and copyable:
    MyClass(const MyClass& rhs);
    MyClass& operator=(const MyClass& rhs);

    void DoSth();
    void DoConst() const;

private:
    // const access:
    const MyClassImpl* Pimpl() const { return m_pImpl.get(); }
    MyClassImpl* Pimpl() { return m_pImpl.get(); }

    std::unique_ptr<MyClassImpl> m_pImpl;
};

cpp_pimpl2.cpp

#include "cpp_pimpl2.h"
#include <iostream>

void* operator new(size_t count) {
    std::cout << "allocating " << count << " bytes\n";
    return malloc(count);
}

void operator delete(void* ptr) noexcept {
    std::cout << "global op delete called\n";
    free(ptr);
}

class MyClassImpl
{
public:
    MyClassImpl(MyClass* pBackPtr) : m_pMainClass(pBackPtr) { }
    ~MyClassImpl() = default;

    void DoSth() { std::cout << "Val (incremented): " << ++m_val << "\n";}

    // try to uncomment (or comment 'const' for the method)
    void DoConst() const { 
        std::cout << "Val: " << /*++*/m_val << "\n"; 
    } 

private:
    int m_val {0};
    MyClass* m_pMainClass {nullptr}; // back pointer
};

MyClass::MyClass() : m_pImpl(new MyClassImpl(this)) 
{
    std::cout <<  __PRETTY_FUNCTION__ << "\n";
}

MyClass::~MyClass() { std::cout <<  __PRETTY_FUNCTION__ << "\n"; }
MyClass::MyClass(MyClass &&) noexcept = default;
MyClass& MyClass::operator=(MyClass &&) noexcept = default;

MyClass::MyClass(const MyClass& rhs)
    : m_pImpl(new MyClassImpl(*rhs.m_pImpl))
{}

MyClass& MyClass::operator=(const MyClass& rhs) {
    if (this != &rhs) 
        m_pImpl.reset(new MyClassImpl(*rhs.m_pImpl));

    return *this;
}

void MyClass::DoSth()
{
    Pimpl()->DoSth();
}

void MyClass::DoConst() const
{
    Pimpl()->DoConst();
}

cpp_pimpl2_client.cpp

#include "cpp_pimpl2.h"
#include <iostream>
#include <vector>

int main()
{
    MyClass myObject;
    myObject.DoSth();

    const MyClass secondObject;
    secondObject.DoConst();
}

As you can see, there is some code here which is a template. That's why there are several approaches to wrapping this idiom in a separate utility class. Let's see below.

As a separate class

For the Herb Sutter example, GotW #101: Compilation Firewalls, Part 2 suggests the following wrapper.

takenFromHerbSutter.h

template<typename T>
class pimpl {
private:
    std::unique_ptr<T> m;
public:
    pimpl();
    template<typename ...Args> pimpl( Args&& ... );
    ~pimpl();
    T* operator->();
    T& operator*();
};

However, you are left with the implementation of the copy constructor if needed.

If you want a full blown wrapper, take a look at this post PIMPL, Rule of Zero and Scott Meyers by Andrey Upadyshev.

In this article, you can see a very advanced implementation of such a helper type:

SPIMPL (Smart Pointer to IMPLementation) is a small header-only C++11 library to simplify the implementation of the PIMPL idiom

Inside the library you can find two types: 1) spimpl::unique_impl_ptr - for pimpl relocatable wrapper only and 2) spimpl::impl_ptr for pimpl relocatable and copyable wrapper.

Fast pimpl

One obvious point about impl is that memory allocation is needed to store the private parts of the class. If you like to avoid it... and you really care about this memory allocation... you can try:

• provide a custom allocator and use some of the fixed memory for a private implementation
• or reserve a large block of memory in the main class and use the new allocation to allocate space for pimpl.
• Note that reserving space up front is flaky - what if the size changes? And more importantly - do you have the correct alignment for the type?

Herb Sutter wrote about this idea here GotW #28: The Fast Pimpl Idiom .

The modern version using the C++11 feature aligned_storage is described here:
My Favourite C++ Idiom: Static PIMPL / Fast PIMPL by Kai Dietrich or Type-safe Pimpl implementation without overhead | Probably Dance blog .

But keep in mind that this is just a trick, it might not work. Or it might work on one platform/compiler but not on another configuration.

In my personal opinion, I don't think this approach is good. Pimp is usually used for larger classes (perhaps managers, types in module interfaces), so the extra cost won't do much.

We have seen several main parts of the pimpl pattern, so now we can discuss its strengths and weaknesses.

Pros and cons

For

• Provides a Compilation Firewall : if a private implementation changes client code, no recompilation is required.
• Headers can get smaller since types only mentioned in the class implementation no longer need to be defined for client code.
• So, in general, it can lead to better compile times.
• Provides Binary Compatibility : very important for library developers. As long as the binary interface remains the same, you can link your application against a different version of the library.
• To simplify, if you add a new virtual method the ABI will change, but adding non-virtual methods (without removing existing ones, of course) does not change the ABI.
• Смотрите Fragile Binary Interface Problem .
• Possible advantage: no v-table (if the main class contains only non-virtual methods).
• Little point: can be used as an object on the stack

Against

• Performance - added one level of indirection.
• A block of memory must be allocated (or pre-allocated) for a private implementation.
• Possible memory fragmentation
• Complex code and requires some discipline to maintain such classes.
• Debugging - you don't see the details right away, the class is split

Other questions

• Testability - There is a perception that when you try to test such a pimpl class, it can cause problems. But generally you only check the public interface, it doesn't matter.
• Not for every class. This pattern is often better suited for large classes at the "interface level". I don't think vector3d with this template would be a good idea...

Alternatives

• Redesign the code
  *To improve build time:
• Use precompiled header files
• Use build caches
• Use incremental build mode
• Abstract interfaces
• Does not provide ABI compatibility, but is a great alternative as a dependency breaking technology
• Gamasutra - In-depth: PIMPL vs pure virtual interfaces
• WITH
• also based on abstract interfaces, but with some more basic mechanisms.

How about modern C++

As of C++17, we don't have any new features dedicated to pimpl. With C++11 we have smart pointers, so try to implement pimpl with smart pointers instead of raw pointers. Plus, of course, we get a lot of metaprogramming stuff to help with declaring wrapper types for the pimpl template.

But in the future, we may want to consider two options: Modules and the dot operator.

Modules will play an important role in reducing compilation time. I haven't played with modules much, but as I see it, using pimpl just for compilation speed may become less critical. Of course, keeping dependencies low is always important.

Another feature that could come in handy is the dot operator, developed by Bjarne Stroustup and Gabriel Dos Reis. PDF - N4477 - not made in C++17, but maybe we will see it in C++20?

Basically, it allows you to overwrite the dot operator and provide much more convenient code for all proxy types.

Who uses

I have collected the following examples:

• QT:
• These are probably the most striking examples (that you can find in the public domain) in which proprietary implementations were widely used.
• There is even a good introductory article discussing d-pointers (as they call pimpl): Pointer - Qt Wiki
• QT also shows how to use pimpl with inheritance. Theoretically, you need a separate pimpl for each derived class, but QT only uses one pointer.
• OpenSceneGraph
• Framebuffer object
• Assimp library
• Exporter
• Take a look at this comment in assimp.hpp :)

// Священный материал, только для членов высшего совета джедаев.
class ImporterPimpl;

// ...

// Просто потому, что мы не хотим, чтобы вы знали, как мы взламываем.
ImporterPimpl* pimpl;

• Open Office
• Для примера laycache.hxx
• Other uses of pimpl
• PhysX or Nvidia

Looks like the pattern is used everywhere :)

Let me know if you have other examples.

If you need more examples, follow these two questions on stack overflow:

• Is the pImpl idiom really used in practice? - Stack Overflow
• Where is pimpl used in C++ apps/libs? any examples? - Stack Overflow

Conclusion

Pimpl looks simple... but as usual in C++ things are not so easy in practice :)

Basic moments:

• Pimpl provides ABI compatibility and reduces compilation dependencies.
• Since C++11, you must use unique_ptr (or even shared_ptr) to implement a template.
• To make the pattern work, decide if your main class should be copyable or just moveable.
• Take care of const methods so that the private implementation honors them.
• If the private implementation requires access to the primary members of the class, then a "backward pointer" is needed.
• Some optimizations are possible (to avoid split memory allocation), but this can be tricky.
• There are many uses of the pattern in open source projects, QT uses it heavily (with inheritance and back-pointer)

Article written by: Bartek | Monday, January 8, 2018