In part one of this series, we established stability as the foundation of any modernization effort. If your system isn’t stable, performance optimizations are essentially meaningless—-after all, a fast crash is still a crash.

But once you’ve built that solid foundation, it’s time to explore one of C++’s greatest strengths: performance. Modern C++ offers an impressive array of tools and techniques that can dramatically improve your application’s speed and resource efficiency without sacrificing the stability we worked so hard to achieve. So many that I decided to break this post into two, a Part A and a Part B.

In Part A, let’s dig into what you can do in your application code–syntax, standard libraries, and best practices. In Part B, we’ll dig into the tools, profiling, and hardware-based techniques available to you in C++.

THE PERFORMANCE CHALLENGE

Performance optimization in C++ has evolved far beyond the traditional advice of “avoid dynamic memory allocation” and “use inline functions.” Today’s hardware and modern C++ features create new optimization opportunities and challenges.

• Processor architecture has shifted from frequency scaling to multi-core and vectorization
• Memory access patterns often matter more than pure computational complexity
• Move semantics and value categories have transformed optimal resource management
• Standard library components have specialized algorithms with sophisticated performance characteristics

The challenge lies in leveraging these modern features without introducing instability or impenetrable complexity. As the saying goes: “Premature optimization is the root of all evil,” but appropriate optimization at the right time with the right tools is still essential.

UNDERSTANDING MODERN PERFORMANCE BOTTLENECKS

Before diving into specific techniques, let’s examine how performance bottlenecks have evolved in modern systems. According to research from Intel’s software division, the relative costs of various operations have shifted dramatically over time.

This growing disparity—known as the “Memory Wall"—means that memory access patterns often dominate performance considerations.

LEVERAGING MOVE SEMANTICS AND VALUE CATEGORIES

One of the most significant performance enhancements in modern C++ is move semantics, introduced in C++11 and refined in subsequent standards. Move semantics allow resources to be transferred between objects without expensive deep copies—providing massive performance gains for objects that manage resources like memory, file handles, or network connections.

Understanding Value Categories: Lvalues and Rvalues

To grasp move semantics, we first need to understand value categories. In C++, every expression belongs to a specific value category that determines how it can be used.

The names derive from where they can appear in an assignment: lvalues can be on the left side of an assignment (they have an address you can assign to), while rvalues can only appear on the right side.

Understanding Rvalue References and Move Semantics

The foundation of move semantics is the rvalue reference (&&), which allows functions to distinguish between lvalues (objects with persistent storage) and rvalues (temporary objects).

Let’s examine a real-world example using a custom string class:

Traditional approach (pre-C++11):

class String {
private:
    char* data;
    size_t length;

public:
    // Copy constructor
    String(const String& other) {
        length = other.length;
        data = new char[length + 1];
        std::memcpy(data, other.data, length + 1);
    }

    // Assignment operator
    String& operator=(const String& other) {
        if (this != &other) {
            delete[] data;
            length = other.length;
            data = new char[length + 1];
            std::memcpy(data, other.data, length + 1);
        }
        return *this;
    }

    // Other methods...
};

// Usage
String createLongString() {
    String result = "Really long string...";
    // Process result...
    return result; // This creates a temporary that must be copied
}

String s = createLongString(); // Copy constructor called

Modern approach with move semantics:

class String {
private:
    char* data;
    size_t length;

public:
    // Copy constructor (unchanged)
    String(const String& other) {
        length = other.length;
        data = new char[length + 1];
        std::memcpy(data, other.data, length + 1);
    }

    // Move constructor
    String(String&& other) noexcept 
        : data(other.data), length(other.length) {
        // Take ownership of resources
        other.data = nullptr;
        other.length = 0;
    }

    // Move assignment operator
    String& operator=(String&& other) noexcept {
        if (this != &other) {
            delete[] data;
            // Take ownership of resources
            data = other.data;
            length = other.length;
            other.data = nullptr;
            other.length = 0;
        }
        return *this;
    }

    // Other methods...
};

// Usage remains the same, but now moves happen instead of copies
String createLongString() {
    String result = "Really long string...";
    // Process result...
    return result; // This creates a temporary that can be moved from
}

String s = createLongString(); // Move constructor called

The performance difference can be dramatic. According to measurements by Bjarne Stroustrup and colleagues, move operations can be more than an order of magnitude faster than copies for resource-heavy objects. The reason is simple: instead of deep-copying the entire data structure (with potentially expensive memory allocations), we simply transfer ownership of the existing resources by swapping or reassigning pointers.

In real-world applications, a benchmark by embeddeduse.com showed that move semantics provided up to 70% performance improvement for operations like shuffling and sorting containers of complex objects. Even for simpler operations, improvements of 20-30% are common when handling resource-managing objects.

Automatic Return Value Optimization (RVO)

Even better than moving is avoiding copies and moves altogether. Modern C++ compilers can eliminate many copies and moves through Return Value Optimization (RVO) and Named Return Value Optimization (NRVO).

String createString() {
    return String("Hello, world"); // No move or copy needed!
}

String createString2() {
    String result("Hello, world");
    return result; // NRVO can eliminate the copy/move
}

C++17 made this guarantee stronger with mandatory copy elision in certain contexts, ensuring that unnecessary copies/moves don’t happen even when they would have observable side effects.

Perfect Forwarding with Universal References

To maximize performance when working with generic code, modern C++ offers “universal references” (also called “forwarding references”) combined with std::forward.

template<typename T>
void wrapper(T&& arg) {
    // Forward arg with its original value category preserved
    processValue(std::forward<T>(arg));
}

What’s special about std::forward is that it preserves the value category of the original argument. This pattern ensures that:

1. If wrapper() is called with an lvalue, std::forward ensures that arg is treated as an lvalue inside processValue()
2. If wrapper() is called with an rvalue, std::forward ensures that arg is treated as an rvalue inside processValue()

Without this mechanism, writing generic code that preserves move semantics would be extremely difficult. Perfect forwarding enables the creation of highly efficient generic containers and algorithms that can take full advantage of move semantics.

DATA STRUCTURES FOR PERFORMANCE

The standard library has evolved significantly, offering containers and algorithms optimized for modern hardware. Choosing the right container for your specific use case can yield dramatic performance improvements.

Choosing the Right Container

Here’s a performance comparison of common operations across standard containers:

* Amortized constant time
** Average case, O(n) worst case

Some container selection guidelines for performance:

1. Prefer std::vector by default - Its cache locality and low overhead make it surprisingly efficient, even for operations where its big-O complexity seems poor
2. Use std::unordered_map instead of std::map for lookups - Unless you need the ordering, hash-based lookups are typically much faster
3. Consider std::array for fixed-size collections - It avoids all dynamic memory operations
4. Be cautious with node-based containers (std::list, std::map) - Their poor cache locality often outweighs their theoretical advantages
5. Use std::string_view and std::span for non-owning views - They provide reference semantics without copying

Small String Optimization and Small Vector Optimization

Modern implementations of std::string typically use the Small String Optimization (SSO), where small strings are stored directly within the string object rather than allocated on the heap. This avoids expensive heap allocations for short strings, which are common in real-world code.

You can implement a similar optimization for vectors with small buffer optimization.

template <typename T, size_t N>
class small_vector {
private:
    // Fixed-size buffer for small collections
    std::aligned_storage_t<sizeof(T), alignof(T)> local_buffer[N];
    std::vector<T> dynamic_buffer;
    
    bool using_local() const { return dynamic_buffer.empty(); }
    
    // Rest of implementation...
};

This approach gives you stack-allocated performance for small collections while maintaining the flexibility of dynamic allocation for larger ones.

Using Custom Allocators for Special Memory Needs

Standard containers allow custom allocators, which can dramatically improve performance for specific use cases.

// Pool allocator for fixed-size objects
template <typename T, size_t BlockSize = 4096>
class pool_allocator {
    // Implementation details...
};

// Usage
std::vector<MyObject, pool_allocator<MyObject>> objects;

Custom allocators can be particularly effective for:

• Objects with specific alignment requirements
• High-frequency allocation/deallocation patterns
• Memory-constrained environments
• Specialized hardware or memory regions

OPTIMIZING MODERN ALGORITHMS

The C++ standard library offers a wealth of optimized algorithms. Using these instead of hand-rolled loops can yield significant performance benefits.

Leveraging the Algorithm Library

Consider this common task: find all elements in a vector that satisfy a condition, then transform them.

Traditional approach:

std::vector values = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
std::vector results;

for (size_t i = 0; i < values.size(); ++i) {
    if (values[i] % 2 == 0) {  // Find even numbers
        results.push_back(values[i] * values[i]);  // Square them
    }
}

While this traditional approach is simple to read, it has potential performance issues:

1. It doesn’t pre-allocate memory for the results vector, potentially causing multiple reallocations
2. Each reallocation can trigger expensive memory copies as the vector grows
3. The code mixes filtering logic with transformation logic, making it harder to separate concerns
4. The algorithmic intent isn’t as clearly expressed as it could be

Modern STL approach:

std::vector values = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
std::vector results;

// Reserve space for efficiency - prevents multiple reallocations
// (Worst case: half the values could be even)
results.reserve(values.size() / 2 + 1);

// First, copy all elements that match our condition
std::copy_if(
    values.begin(), values.end(),    // Source range
    std::back_inserter(results),     // Destination (appends to results)
    [](int x) { return x % 2 == 0; } // Predicate function
);

// Then transform all elements in-place by applying the square function
std::transform(
    results.begin(), results.end(),  // Source range
    results.begin(),                 // Destination (overwrite in-place)
    [](int x) { return x * x; }      // Transformation function
);

This modern approach provides multiple advantages:

• reserve() pre-allocates memory, avoiding costly reallocations
• std::back_inserter creates an iterator that calls push_back() for us
• std::copy_if and std::transform clearly separate the filtering from the transformation
• The algorithm names explicitly state our intentions, making the code more self-documenting
• The compiler has more optimization opportunities with standard algorithms

C++20 and newer approaches:

std::vector values = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
std::vector results;
results.reserve(values.size() / 2 + 1);

// C++23 approach (combines filtering and transformation in one pass)
std::transform_if(
    values.begin(), values.end(),    // Source range
    std::back_inserter(results),     // Destination
    [](int x) { return x * x; },     // Transformation function
    [](int x) { return x % 2 == 0; } // Predicate function
);

// C++20 ranges approach (declarative pipeline style)
auto even_numbers = values | std::views::filter([](int x) { return x % 2 == 0; });
auto squared = even_numbers | std::views::transform([](int x) { return x * x; });
results.assign(squared.begin(), squared.end());

The C++20 ranges approach is particularly powerful because:

• It creates a processing pipeline that clearly shows the data flow
• Operations are only applied when needed (lazy evaluation)
• It avoids creating intermediate collections between steps
• The code reads like a clear sequence of operations to perform

While the modern approaches may initially look more verbose, their benefits include:

1. Better performance through pre-allocation and optimized algorithms
2. Clearer expression of intent through named algorithms
3. Separation of concerns (filtering vs. transforming)
4. Fewer opportunities for subtle bugs
5. Better compiler optimization opportunities

In performance-critical code, these algorithm-based approaches can be significantly faster, especially as data sizes grow. For a vector of 1 million elements, the modern approach with proper reserving can be 2-3x faster than the naive loop approach due to fewer allocations and better memory access patterns.

Parallel Algorithms (C++17 and Beyond)

C++17 introduced parallel versions of many standard algorithms, making it trivial to leverage multi-core processors.

std::vector values(10'000'000);
// Fill vector...

// Sequential sort
std::sort(values.begin(), values.end());  

// Parallel sort - potentially much faster on multi-core systems
std::sort(std::execution::par, values.begin(), values.end());

// Parallel unsequenced sort - allows even more optimization opportunities
std::sort(std::execution::par_unseq, values.begin(), values.end());

Measurements show these parallel algorithms can achieve near-linear speedups on multi-core systems for suitable workloads. The Intel TBB library, which often underlies these implementations, has shown speedups of 4-8x on 8-core systems for common operations.

When using parallel algorithms, it’s essential to ensure that your operations are thread-safe. The standard library algorithms handle their internal synchronization, but if your provided functors access shared state, you’ll need to provide proper synchronization mechanisms.

MODERN STRING HANDLING

String operations are often performance bottlenecks in C++ applications. C++17 and higher offers several improvements.

String Views for Non-owning References

Introduced in C++17, std::string_view provides a non-owning reference to a string, eliminating unnecessary copies:

Old approach:

bool startsWith(const std::string& str, const std::string& prefix) {
    return str.size() >= prefix.size() &&
           str.compare(0, prefix.size(), prefix) == 0;
}

// Usage - creates temporary std::string objects
if (startsWith(some_string, "http://")) {
    // ...
}

Modern approach:

bool startsWith(std::string_view str, std::string_view prefix) {
    return str.size() >= prefix.size() &&
           str.compare(0, prefix.size(), prefix) == 0;
}

// Usage - no temporary objects created
if (startsWith(some_string, "http://")) {
    // ...
}

Format Library (C++20)

C++20 introduces a type-safe formatting library that’s both more convenient and potentially more efficient than std::stringstream or sprintf.

// Instead of:
std::string message;
{
    std::ostringstream oss;
    oss << "User " << user_id << " logged in at " << timestamp;
    message = oss.str();
}

// Or:
char buffer[100];
sprintf(buffer, "User %d logged in at %s", user_id, timestamp.c_str());
std::string message(buffer);

// Use:
std::string message = std::format("User {} logged in at {}", user_id, timestamp);

The format library outperforms stringstream by avoiding temporary allocations and complex locale handling, while providing complete type safety unlike sprintf. According to benchmarks in Aras Pranckevičius’s blog, std::format can be significantly faster than stringstream and has better scaling with multiple threads.

The format library also supports a wide range of formatting options with a Python-inspired syntax.

// Format integers with different bases
std::string hex = std::format("{:x}", 42);    // "2a"
std::string oct = std::format("{:#o}", 42);   // "052"

// Format floating point with precision
std::string pi = std::format("{:.3f}", 3.14159);  // "3.142"

// Format with alignment and width
std::string right = std::format("{:>10}", "text");  // "      text"
std::string left = std::format("{:<10}", "text");   // "text      "

// Format with custom fill character
std::string fill = std::format("{:*^10}", "text");  // "***text***"

For even better performance, C++20 also offers std::format_to, which allows writing directly to a pre-allocated buffer:

std::array<char, 100> buffer;
auto result = std::format_to(buffer.data(), "User {} logged in at {}", user_id, timestamp);
std::string_view message(buffer.data(), result - buffer.data());

OPTIMIZING MEMORY USAGE AND LAYOUT

Memory layout plays a crucial role in modern C++ performance. Here are some key techniques.

Structure of Arrays vs Array of Structures

Traditional object-oriented programming encourages grouping related data into a single structure.

// Array of Structures (AoS)
struct Particle {
    Vector3 position;
    Vector3 velocity;
    float mass;
};

std::vector<Particle> particles(10000);

// Process particles
for (const auto& p : particles) {
    // Work with p.position, p.velocity, p.mass
}

However, for performance-critical code, the “Structure of Arrays” approach often performs better due to improved cache locality.

// Structure of Arrays (SoA)
struct ParticleSystem {
    std::vector<Vector3> positions;
    std::vector<Vector3> velocities;
    std::vector<float> masses;
};

ParticleSystem particles;
particles.positions.resize(10000);
particles.velocities.resize(10000);
particles.masses.resize(10000);

// Process just positions and velocities
for (size_t i = 0; i < particles.positions.size(); ++i) {
    particles.positions[i] += particles.velocities[i];
}

The SoA approach can yield 2-4x performance improvements for operations that only need a subset of the data, due to better cache utilization. It’s particularly effective for SIMD vectorization (which we’ll cover in part 2b).

Custom Memory Alignment for Performance

Modern processors are sensitive to memory alignment, especially when using SIMD instructions.

// Unaligned structure
struct Matrix {
    float data[16];  // 4x4 matrix
};

// Aligned for SIMD operations
struct alignas(32) AlignedMatrix {
    float data[16];  // 4x4 matrix
};

For certain operations, properly aligned data can be 2-3x faster than unaligned data, as it allows direct use of aligned SIMD load/store instructions.

Avoiding False Sharing in Multithreaded Code

“False sharing” occurs when two threads access different variables that happen to be on the same cache line, causing cache coherency traffic:

// Potential false sharing
struct ThreadData {
    std::atomic<int> count1;  // Thread 1 updates this
    std::atomic<int> count2;  // Thread 2 updates this
};

// Avoid false sharing
struct PaddedData {
    alignas(std::hardware_destructive_interference_size) std::atomic<int> count1;
    alignas(std::hardware_destructive_interference_size) std::atomic<int> count2;
};

C++17 introduced std::hardware_destructive_interference_size specifically to help address this issue, allowing you to pad data structures appropriately.

Benchmarks from Intel’s Threading Building Blocks library show that eliminating false sharing can improve performance by 10x or more in thread-intensive applications.

INLINING AND FUNCTION CALL OPTIMIZATION

Function call overhead has traditionally been a concern in C++. There are a few nuanced approaches available in C++17 and higher, thought the built-in intelligence in the compiler can usually do a better job with JIT optimization.

Explicit Inlining vs Compiler Decisions

While the inline keyword exists, modern compilers make their own decisions about inlining based on heuristics:

// Suggestion to inline
inline int add(int a, int b) {
    return a + b;
}

// Force inlining (compiler-specific)
__forceinline int forceAdd(int a, int b) {
    return a + b;
}

// Prevent inlining (standardized in C++20)
[[noinline]] int noInlineAdd(int a, int b) {
    return a + b;
}

It’s usually best to let the compiler decide on inlining, as it can make more informed decisions based on the broader context.

Optimizing Virtual Function Calls

Virtual function calls can impact performance due to the indirect call through the vtable. When polymorphism is needed, consider these optimizations:

1. Use final when appropriate:

class Base {
    virtual void method() { /* ... */ }
};

class Derived final : public Base {
    void method() final { /* ... */ }
};

2. Consider CRTP for static polymorphism:

template <typename Derived>
class Base {
public:
    void interface() {
        // Call the derived implementation
        static_cast<Derived*>(this)->implementation();
    }
};

class ConcreteType : public Base<ConcreteType> {
public:
    void implementation() {
        // Concrete implementation
    }
};

Virtual function overhead is typically small, but in tight loops or performance-critical code, these techniques can yield measurable improvements. Benchmarks from Quick C++ Benchmark show virtual function calls being 5-10% slower than direct calls and CRTP implementations in typical cases.

LEVERAGING CONSTEXPR AND COMPILE-TIME COMPUTATION

C++11 and higher allows more computation to happen at compile time, and, in some cases, eliminating runtime overhead entirely.

Compile-time Function Evaluation

// Compute values at compile time
constexpr int fibonacci(int n) {
    if (n <= 1) return n;
    return fibonacci(n-1) + fibonacci(n-2);
}

// Usage
constexpr int result = fibonacci(20);  // Computed at compile time

C++20 greatly expanded what’s possible in constexpr functions, allowing dynamic memory allocation, try/catch, and more.

Template Metaprogramming Simplified

C++11 and beyond have made template metaprogramming much more approachable:

// C++98/03 style
template <unsigned N>
struct Factorial {
    static const unsigned value = N * Factorial<N-1>::value;
};

template <>
struct Factorial<0> {
    static const unsigned value = 1;
};

// C++14 and beyond
template <unsigned N>
constexpr unsigned factorial() {
    if (N == 0) return 1;
    return N * factorial<N-1>();
}

These modern approaches are not only more readable, but often compile faster as well.

Immediate Functions (C++20)

C++20 introduced consteval, which creates functions that must be evaluated at compile time.

consteval int sqr(int n) {
    return n * n;
}

// This will be a compile-time constant
int x = sqr(100);

// This would be an error - cannot be evaluated at compile time
// int y = sqr(runtime_value);

This feature ensures that certain computations happen at compile time, preventing inadvertent runtime costs.

CONCLUSION AND NEXT STEPS

In this first part of our performance optimization journey, we’ve focused on code-level improvements that leverage modern C++ features:

• Move semantics and value categories to eliminate unnecessary copies
• Modern container selection for optimal access patterns
• Algorithm library and ranges for expressive, efficient code
• Memory layout optimization for cache-friendly access
• Compile-time computation to eliminate runtime costs

These techniques allow you to write code that’s not only faster but often clearer and more maintainable as well. The beauty of modern C++ is that many of these optimizations align with good software engineering practices, encouraging composable, expressive code that’s also highly efficient.

In Part B, we’ll explore the next layer of performance optimization:

• Tools and techniques for measuring performance
• Compiler optimizations and how to enable them
• Profiling techniques to identify bottlenecks
• SIMD vectorization for data-parallel operations
• Platform-specific optimizations

Remember that the foundation of performance optimization is stability. As you apply these techniques, be sure to maintain your comprehensive testing strategy to ensure that your optimizations don’t introduce new bugs or vulnerabilities.

The key to successful modernization is balancing short-term improvements with long-term maintainability—choosing optimizations that not only make your code faster today but also easier to understand, extend, and maintain tomorrow.