Iterators are a fundamental concept in the STL. Iterators are implemented with the semantics of C pointers, using the same increment, decrement, and dereference operators. The pointer idiom is already familiar and it allows algorithms such as std::sort and std::transform to work with both STL containers and primitive memory buffers.
This chapter contains the following recipes:
Before we dive into the recipes, let's first take a quick look at how iterators work.
The STL uses iterators to navigate the elements of its container classes. Most containers include begin() and end() iterators. These are usually implemented as member functions that return an iterator object. The begin() iterator points to the initial container element, and the end() iterator points past the final element:
Figure 4.1: The begin() and end() iterators
The end() iterator may function as a sentinel to mark the ending boundary for containers of indeterminate length. We'll see some examples of that in this chapter.
Most STL containers define their own specific iterator type. For example, for a vector of int :
std::vector<int> v; The iterator type would be defined as:
std::vector<int>::iterator v_it; You can see how this could easily get out of hand. If we had a vector of vector of string :
std::vector<std::vector< std::string>> v; Its iterator type would be:
std::vector<std::vector< std::string>>::iterator v_it; Fortunately, C++11 gave us automatic type deduction and the auto type. By using auto , we rarely need to use the full iterator type definition. For example, if we need an iterator in a for loop, we can use the auto type:
for (auto v_it = v.begin(); v_it != v.end(); ++v_it) { println("{}", *v_it); } Notice the use of the dereference operator * to access the elements from the iterator. This is the same syntax you would use to dereference a pointer:
const int a[] {1, 2, 3, 4, 5}; size_t count {sizeof(a) / sizeof(int)}; for (const int* p = a; count > 0; ++p, --count) { println("{}", *p); } This also means that you can use a range-based for loop with either a primitive array:
const int a[] {1, 2, 3, 4, 5}; for (auto e : a) { println("{}", e); } Or with an STL container:
std::vector<int> v {1, 2, 3, 4, 5}; for (auto e : v) { println("{}", e); } The range-based for loop is just a shorthand for a C-style for loop with iterators:
auto begin_it = std::begin(v); auto end_it = std::end(v); for (; begin_it != end_it; ++begin_it) { auto e = *begin_it; println("{}", e); } Because iterators use the same syntax as a primitive pointer, the range-based for loop works the same with either container.
Notice that the range-based for loop calls std::begin() and std::end() , instead of directly calling the begin() and end() member functions. The std:: functions call the member functions to get the iterators. So, why not just call the member functions? The std:: non-member functions are designed to also work with primitive arrays. That's why a for loop works with an array:
const int arr[] {1, 2, 3, 4, 5}; for (auto e : arr) { println("{}", e); } Output:
1 2 3 4 5 For most purposes, I tend to favor the member functions begin() and end() because they are more explicit. Others favor the std:: non-member functions because they are more general. Six or half a dozen; I suggest you pick a style and stick with it.
Prior to C++20, iterators were divided into categories based on their capabilities:
Table 4.1: Iterator categories
These categories are hierarchical, where the more capable iterators inherit the capabilities of the less capable iterators. In other words, the input iterator can read and increment once. The forward iterator has the capabilities of the Input Iterator plus it can increment multiple times. The bidirectional iterator has those capabilities plus it can decrement. And on down the list.
The output iterator can write and increment once. If any of the other iterators can also write, it is considered a mutable iterator .
While the pre-C++20 iterator categories remain valid, C++20 introduced concepts and constraints , which provide compile-time semantics for clearer diagnostics . A concept is simply a named constraint that restricts the types of arguments to a template function or class and helps the compiler choose appropriate specializations.
Beginning with C++20, the STL defines iterators in terms of concepts instead of categories. Each of these concepts is in the std:: namespace.
Table 4.2: C++20 iterator concepts
You can use these concepts to constrain the arguments of a template:
template<typename T> requires std::random_access_iterator<typename T::iterator> void printc(const T & c) { for(auto e : c) { println("{}", e); } print("\n"); println("element 0: {}", c[0]); } This function requires a random_access_iterator . If we call it with a list , which is not a random-access container, the compiler will give us an error.
int main() { list<int> c {1, 2, 3, 4, 5}; printc(c); } The list iterator type does not support the random_access_iterator concept. So, the compiler gives me an error:
error: no matching function for call to 'printc(std::__cxx11::list<int>&)' 27 | printc(c); | ~~~~~~^~~ note: candidate: 'template<class T> requires random_access_iterator<typename T::iterator> void printc(const T&)' 16 | void printc(const T & c) { | ^~~~~~ note: template argument deduction/substitution failed: note: constraints not satisfied This is the error output from GCC. Your errors may look different.
If I call the same template function with a vector , which is a random-access container:
int main() { vector<int> c{ 1, 2, 3, 4, 5 }; printc(c); } Now it compiles and runs without errors:
$ ./working 1 2 3 4 5 element 0: 1 While there are different types of iterators for different types of capabilities (and concepts), the complexity is there to support ease of use.
By using these concepts, diagnostics are moved from run time to the compiler, resulting in smaller, more efficient, and more secure code.
This recipe describes a simple class that generates an iterable range, suitable for use with the range-based for loop. The idea is to create a sequence generator that iterates from a beginning value to an ending value.
To accomplish this task, we need an iterator class, along with the object interface class.
There are two major parts to this recipe: the main interface class Seq , and a nested iterator class.
Seq class. It only needs to implement the begin() and end() member functions: template<typename T> class Seq { T seq_start {}; T seq_end {}; public: Seq(T start, T end) : seq_start {start}, seq_end {end} {} iterator begin() const { return iterator {seq_start}; } iterator end() const { return iterator {seq_end}; } }; The constructor sets up the seq_start and seq_end variables. These are used to construct the begin() and end() iterators, respectively. The member functions begin() and end() return iterator objects.
iterator class is normally defined inside the public section of the container class. This is called a member class or a nested class . We'll insert it right after the Seq constructor: class iterator { T it_value{}; public: explicit iterator(T position = 0) : it_value {position} {} T operator*() const { return it_value; } iterator& operator++() { ++it_value; return *this; } bool operator!=(const Seq::iterator& other) const { return it_value != other.it_value; } }; It's traditional to name the iterator class iterator . This allows it to be referenced as Seq< type >::iterator .
The iterator constructor is qualified explicit to avoid implicit conversions.
The it_value variable is maintained by the iterator. This is used to return a value from the pointer dereference.
The minimum requirement for supporting the range-based for loop is a dereference operator * , a pre-increment operator ++ , and the not-equal comparison operator != .
main() function to test our sequence generator: int main() { Seq<int> r(100, 110); for (auto v : r) { print("{} ", v); } print("\n"); } This constructs a Seq object and prints out its sequence.
The output looks like this:
$ ./seq 100 101 102 103 104 105 106 107 108 109 The point of this recipe is to make a sequence generator that works with a range-based for loop. Let's first consider the equivalent code for the range-based for loop:
{ auto begin_it = std::begin(container); auto end_it = std::end(container); for (; begin_it != end_it; ++begin_it){ auto e = *begin_it; println("{}", e); } } From this equivalent code, we can deduce the minimum requirements for an object to support the range-based for loop:
begin() and end() iterators != operator ++ operator * operator Our main Seq class interface only has three public member functions: the constructor, and the begin() and end() iterators:
Seq(T start, T end) : seq_start {start}, seq_end {end} {} iterator begin() const { return iterator {seq_start}; } iterator end() const { return iterator {seq_end}; } The implementation of the Seq::iterator class carries the actual payload:
class iterator { T it_value{}; This is the common configuration because the payload is only accessed through iterators.
We've implemented only the three operators we need:
T operator*() const { return it_value; } iterator& operator++() { ++it_value; return *this; } bool operator!=(const Seq::iterator& other) const { return it_value != other.it_value; } This is all we need to support the range-based for loop:
Seq<int> r(100, 110); for (auto v : r) { print("{} ", v); } print("\n"); Output:
100 101 102 103 104 105 106 107 108 109 It's traditional, but not required, to define the iterator as a member class of the container. This allows the iterator type to be subordinate to the container type:
Seq<int>::iterator it = r.begin(); This is less important post-C++11 because of the auto type, but it's still considered best practice.
Many STL algorithms require iterators to conform to certain traits. Unfortunately, these requirements may be inconsistent across compilers, systems, and C++ versions.
For our purposes, we'll use the class from the Create an iterable range recipe to illustrate the issue. You may find this makes more sense if you read that recipe before continuing.
In main() , if I add a call to the std::minmax_element() algorithm:
Seq<int> r {100, 110}; auto [min_it, max_it] = minmax_element(r.begin(), r.end()); print("{} - {}\n", *min_it, *max_it); This does not compile. The error messages are vague, cryptic, and cascading, but if you look closely, you'll see that our iterator does not meet the requirements to be compatible with this algorithm.
Okay, let's fix that.
The gcc compiler gives a cyrptic error like:
In instantiation of 'constexpr std::pair<_FIter, _FIter> std::__minmax_element(_ForwardIterator, _ForwardIterator, _Compare) [with _ForwardIterator = Seq<int>::iterator; _Compare = __gnu_cxx::__ops::_Iter_less_iter]':
…followed by a cascade of other errors. You can see that it has something to do with the _ForwardIterator token. This is a clue, but not very helpful unless you're familiar with this issue.
We need to make a few simple additions to our iterator to make it compatible with the algorithm. Our iterator needs to meet the minimum requirements for a forward iterator , so let's start there:
== . We can easily add this to our iterator with an operator==() overload: bool operator==(const iterator& other) const noexcept { return it_value == other.it_value; } Interestingly, this makes the code compile and run on some systems, but not on Clang , where we get the error message:
No type named 'value_type' in 'std::iterator_traits<Seq<int>::iterator>' This tells me that we need to set up the traits in the iterator.
iterator_traits class looks for a set of type definitions (implemented as using aliases) in the iterator class: public: using iterator_concept = std::forward_iterator_tag; using iterator_category = std::forward_iterator_tag; using value_type = std::remove_cv_t<T>; using difference_type = std::ptrdiff_t; using pointer = const T*; using reference = constT&; I tend to put these at the top of the public: section of the iterator class, where they'll be easy to find.
This gives us a fully conforming forward iterator class, and the code runs on all the compilers I have.
The using statements are traits that may be used to define what capabilities the iterator can perform. Let's look at each of them:
using iterator_concept = std::forward_iterator_tag; using iterator_category = std::forward_iterator_tag; The first two are the category and the concept , and both are set to forward_iterator_tag . This value indicates that the iterator conforms to the forward iterator specification.
Some code doesn't look at those values, and instead looks for individual settings and capabilities:
using value_type = std::remove_cv_t<T>; using difference_type = std::ptrdiff_t; using pointer = const T*; using reference = constT&; The value_type alias is set to std::remove_cv_t<T> , which is the type of the value, with any const qualifier removed.
The difference_type alias is set to std::ptrdiff_t , a special type for pointer differences.
The pointer and reference aliases are set to const -qualified versions of the pointer and reference, respectively.
Defining these type aliases is a basic requirement for most iterators.
It's worth noting that defining these traits allows us to use concept-restricted templates with our iterator. For example:
template<typename T> requires std::forward_iterator<typename T::iterator> void printc(const T & c) { for(auto v : c) { print("{} ", v); } print("\n"); } This function that prints our sequence is restricted by the forward_iterator concept. If our class did not qualify, it wouldn't compile.
We can also use the ranges:: versions of the algorithms:
auto [min_it, max_it] = ranges::minmax_element(r); This makes it more convenient to use our iterators.
We can test for forward_range compatibility with a static assertion:
static_assert(ranges::forward_range<Seq<int>>); An iterator is essentially an abstraction. It has a specific interface and is used in specific ways. But beyond that, it's just code and it can be used for other purposes. An iterator adapt o r is a class that looks like an iterator but does something else.
The STL comes with an assortment of iterator adaptors. Often used with the algorithm library, they are quite useful. The STL iterator adaptors generally fall into three categories:
Insert iterators , or inserters , are used to insert elements into a container.
Stream iterators read from and write to a stream.
Reverse iterators reverse the direction of an iterator.
We will briefly cover each of these in this recipe.
In this recipe, we'll consider a few examples of STL iterator adaptors.
void printc(const auto & v, const string_view s = "") { if (!s.empty()) print("{}: ", s); for (auto e : v) print("{} ", e); print("\n"); } The printc() function allows us to easily view the results of our algorithms. It includes an optional string_view argument for a description.
main() function, we'll define a couple of deque containers. We're using deque containers so we can insert at both ends: int main() { deque<int> d1 {1, 2, 3, 4, 5}; deque<int> d2(d1.size()); copy(d1.begin(), d1.end(), d2.begin()); printc(d1); printc(d2, "d2 after copy"); } Output:
1 2 3 4 5 d2 after copy: 1 2 3 4 5 We defined deque d1 with five int values, and d2 with space for the same number of elements. The copy() algorithm will not allocate space, so d2 must have room for the elements.
The copy() algorithm takes three iterators: the begin and end iterators indicate the range of elements to copy from, and the begin iterator of the destination range. It does not check the iterators to make sure they're valid. (Try this without allocating space in a vector and you should get a segmentation fault error.)
We call printc() on both containers to show the results.
copy() algorithm is not always convenient for this. Sometimes you want to copy and add elements to the end of a container. It would be nice to have an algorithm that calls push_back() for each element. This is where an iterator adaptor is useful. Let's add some code at the end of main() : copy(d1.begin(), d1.end(), std::back_inserter(d2)); printc(d2, "d2 after back_inserter"); Output:
d2 after back_inserter: 1 2 3 4 5 1 2 3 4 5 back_inserter() is an insert iterator adaptor that calls push_back() for each item assigned to it. You can use it anywhere an output iterator is expected.
front_inserter() adaptor for when you want to insert at the front of a container: deque<int> d3 {47, 73, 114, 138, 54}; copy(d3.begin(), d3.end(), std::front_inserter(d2)); printc(d2, "d2 after front_inserter"); Output:
d2 after front_inserter: 54 138 114 73 47 1 2 3 4 5 1 2 3 4 5 The front_inserter() adaptor inserts elements at the front using the container's push_front() method. Notice that the elements in the destination are reversed, because each element is inserted before the previous one.
inserter() adaptor: auto it2 = d2.begin() + 2; copy(d1.begin(), d1.end(), std::inserter(d2, it2)); printc(d2, "d2 after middle insert"); Output:
d2 after middle insert: 54 138 1 2 3 4 5 114 73 47 ... The inserter() adaptor takes an iterator for the insertion begin point.
iostream objects. This is ostream_iterator() : print("ostream_iterator: "); copy(d1.begin(), d1.end(), ostream_iterator<int>(cout)); print("\n"); Output:
ostream_iterator: 12345 istream_iterator() : vector<string> vs{}; copy(istream_iterator<string>(cin), istream_iterator<string>(), back_inserter(vs)); printc(vs, "vs from istream"); Output:
$ ./working < five-words.txt vs from istream: this is not a haiku The istream_iterator() adaptor will return an end iterator by default, if no stream is passed.
rbegin() and rend() : for(auto it = d1.rbegin(); it != d1.rend(); ++it) { print("{} ", *it); } print("\n"); Output:
5 4 3 2 1 The reverse adaptors, rbegin() and rend() , simply iterate in reverse. This is handy when you need to iterate a container from the end to the beginning.
Iterator adaptors work by wrapping around an existing container. When you call an adaptor, like back_inserter() with a container object:
copy(d1.begin(), d1.end(), back_inserter(d2)); The adaptor returns an object that mimics an iterator, in this case a std::back_insert_iterator object, which calls the push_back() method on the container object each time a value is assigned to the iterator. This allows the adaptor to be used in place of an iterator, while performing its useful task.
The istream_adaptor() also requires a sentinel . A sentinel signals the end of an iterator of indeterminate length. When we read from a stream, we don't know how many objects are in the stream until we hit the end. When the stream hits the end, the sentinel will compare equal with the iterator, which signals the end of the stream. The istream_adaptor() will create a sentinel when it's called without a parameter:
auto it = istream_adaptor<string>(cin); auto it_end = istream_adaptor<string>(); // creates sentinel This allows us to test for the end of a stream, as we would with any container:
for (auto it = istream_iterator<string>(cin); it != istream_iterator<string>(); ++it) { print("{} ", *it); } print("\n"); Output:
$ ./working < five-words.txt this is not a haiku A reverse iterator adaptor is an abstraction that reverses the direction of an iterator class. It requires a bidirectional iterator.
Most bidirectional containers in the STL include a reverse iterator adaptor. Other containers, such as the primitive C-array, do not. Let's look at some examples:
printc() function, as we've used throughout this chapter: void printc(const auto & c, const string_view s = "") { if (s.size()) print("{}: ", s); for (auto e : c) print("{} ", e); print("\n"); } This uses a range-based for loop to print the elements of a container.
for loop works even with primitive C-arrays, which have no iterator class. So, our printc() function already works with a C-array: int main() { int array[] {1, 2, 3, 4, 5}; printc(array, "c-array"); } We get this output:
c-array: 1 2 3 4 5 begin() and end() iterator adaptors to create normal forward iterators for the C-array: auto it = std::begin(c); auto end_it = std::end(c); while (it != end_it) { print("{} ", *it++); } Output from the while loop:
1 2 3 4 5 rbegin() and rend() reverse iterator adaptors to create reverse iterators for the C-array: auto it = std::rbegin(c); auto end_it = std::rend(c); while (it != end_it) { print("{} ", *it++); } Now our output is reversed:
5 4 3 2 1 printc() that prints in reverse: void printr(const auto & c, const string_view s = "") { if (s.size()) print("{}: ", s); auto it = std::rbegin(c); auto end_it = std::rend(c); while (it != end_it) { print("{} ", *it++); } print("\n"); } When we call it with the C-array:
printr(array, "rev c-array"); We get this output:
rev c-array: 5 4 3 2 1 vector<int> v {1, 2, 3, 4, 5}; printc(v, "vector"); printr(v, "rev vector"); Output:
vector: 1 2 3 4 5 rev vector: 5 4 3 2 1 A normal iterator class has a begin() iterator that points to the first element, and an end() iterator that points past the last element:
Figure 4.2: Forward iterator
You iterate the container by incrementing the begin() iterator with the ++ operator, until it reaches the value of the end() iterator.
A reverse iterator adaptor intercepts the iterator interface and turns it around so the begin() iterator points to the last element, and end() iterator points before the first element. The ++ and ‑‑ operators are also inverted:
Figure 4.3: Reverse iterator adaptor
In the reversed iterator, the ++ operator decrements and the ‑‑ operator increments.
It's worth noting that most bidirectional STL containers already include a reverse iterator adaptor, accessible by member functions rbegin() and rend() :
vector<int> v; it = v.rbegin(); it_end = v.rend(); These iterators will operate in reverse and are suitable for many purposes, such as prioritized lists, sortable displays, or any context where you need to traverse a data set in descending order.
Some objects don't have a specific length. To know their length, you need to iterate through all their elements. For example, elsewhere in this chapter we've seen a generator that doesn't have a specific length. A more common example would be a C-string.
A C-string is a primitive C-array of characters, terminated with a null '\0' value.
Figure 4.4: A C-string with its null terminator
We use C-strings all the time, even if we don't realize it. Any literal string in C/C++ is a C-string:
std::string s {"string"}; Here, the STL string s is initialized with a literal string. The literal string is a C-string. If we look at the individual characters in hexadecimal, we'll see the null terminator:
for (char c : "string") { print("{:02x} ", c); } We use the format string {:02x} to print the character in hexadecimal because the null terminator would be invisible when printed as a character.
The word "string" has six letters. The output from our loop shows seven elements in the array:
73 74 72 69 6e 67 00 The seventh element is the null terminator .
The loop sees the primitive C-array of characters, with seven values. The fact that it's a string is an abstraction invisible to the loop. If we want the loop to treat it like a string, we'll need an iterator and a sentinel .
A sentinel is an object that signals the end of an iterator of indeterminate length. When the iterator hits the end of the data, the sentinel will compare equal to the iterator.
To see how this works, let's build an iterator for C-strings!
To use a sentinel with a C-string, we need to build a custom iterator. It doesn't need to be complicated, just the essentials for use with a range-based for loop: a dereference operator * , a pre-increment operator ++ , and the not-equal comparison operator != .
using sentinel_t = const char; constexpr sentinel_t nullchar {'\0'}; The using alias for sentinel_t is const char . We'll use this for the sentinel in our class.
We also define the constant nullchar for the null character terminator.
class cstr_it { const char *s {}; public: explicit cstr_it(const char *str) : s {str} {} char operator*() const { return *s; } cstr_it& operator++() { ++s; return *this; } bool operator!=(sentinel_t) const { return s != nullptr&&*s!=nullchar; } cstr_it begin() const { return *this; } sentinel_t end() const { return nullchar; } }; This is short and simple. It's the minimum necessary for a range-based for loop. Notice the end() function returns a nullchar and the operator!=() overload compares against the nullchar . That's all we need for the sentinel.
void print_cstr(const char * s) { print("{}: ", s); for (char c : cstr_it(s)) { print("{:02x} ", c); } print("\n"); } In this function we first print the string. Then we print each individual character as a hexadecimal value, using the cstr_it iterator to traverse the string.
print_cstr() from our main() function: int main() { const char carray[] {"array"}; print_cstr(carray); const char * cstr {"c-string"}; print_cstr(cstr); } The output looks like this:
array: 61 72 72 61 79 c-string: 63 2d 73 74 72 69 6e 67 Notice that there are no extraneous characters and no null terminators. This is because our sentinel tells the for loop to stop when it sees the nullchar sentinel.
The sentinel part of the iterator class is very simple. We can easily use the null terminator as the sentinel value by returning it in the end() function:
sentinel_t end() const { return nullchar; } Then the not-equal comparison operator can test for it:
bool operator!=(sentinel_t) const { return s != nullptr&&*s!=nullchar; } Notice that the parameter is just a type ( sentinel_t ). A parameter type is necessary for the function signature, but we don't need the value. All that's necessary is to compare the current iterator with the sentinel.
This technique should be useful whenever you have a type or class that doesn't have a predetermined endpoint for comparison.
A generator is an iterator that generates a sequence of values. It does not use a container. It creates values on the fly, returning one value at a time as needed. A C++ generator stands on its own; it does not need to wrap around another object.
In this recipe, we'll build a generator for a Fibonacci sequence . This is a sequence where each number is the sum of the previous two numbers in the sequence, starting with 0 and 1:
Figure 4.5: Definition of a Fibonacci sequence
The first ten values of the Fibonacci sequence, not counting zero, are: 1, 1, 2, 3, 5, 8, 13, 21, 34, 55. This is a close approximation of the golden ratio found in nature.
A Fibonacci sequence is often created with a recursive loop . Recursion in a generator can be difficult and resource-intensive, so instead we just save the previous two values in the sequence and add them together. This approach is more efficient.
void printc(const auto & v, const string_view s = "") { if (!s.empty()) print("{}: ", s); for (auto e : v) print("{} ", e); print("\n"); } We've used this printc() function before. It prints an iterable range, along with a description string, if provided.
private section template<typename T> class fib_generator { T stop {}; T count {}; T n1 {0}; T n2 {1}; The stop variable will be used later as a sentinel . It's set to the number of values to generate. count is used to keep track of how many values we've generated. n1 and n2 are the previous two sequence values, used for calculating the next value.
private scope, we add a simple function for calculating the next value in the Fibonacci sequence constexpr void do_fib() { const T prev_n2 = n2; n2 += n1; n1 = prev_n2; } public scope, we have a simple constructor with a default value: public: explicit fib_generator(T fib_stop = 0) : stop {fib_stop} {} This constructor is used without an argument to create a sentinel. The fib_stop argument initializes the stop variable to hold the number of values to generate.
begin() and end() iterator functions: T operator*() const { return n2; } constexpr fib_generator& operator++() { do_fib(); ++count; return *this; } fib_generator operator++(int) { auto temp = *this; ++*this; return temp; } bool operator!=(const fib_generator& o) const { return count != o.count; } bool operator==(const fib_generator& o) const { return count == o.count; } const fib_generator& begin() const { return *this; } const fib_generator end() const { auto sentinel = fib_generator(); sentinel.count = stop; return sentinel; } fib_t size() { return stop; } Though the post-increment ++ and equality comparison == operators are not required for this example, it's good practice to include them. These are required when the iterator is used with the algorithms library.
There's also a simple size() function which can be useful if you need to initialize a target container for a copy operation.
printc() : int main() { printc(fib_generator(10)); } This creates an anonymous fib_generator object to pass to the printc() function.
10 Fibonacci numbers, not including zero: 1 1 2 3 5 8 13 21 34 55 The fib_generator class operates as a forward iterator, simply because it provides all the necessary interface functions:
template<typename T> fib_generator { public: T operator*() const; constexprfib_generator&operator++(); fib_generator operator++(int); bool operator!=(constfib_generator&o)const; bool operator==(constfib_generator&o)const; const fib_generator& begin() const; const fib_generator end() const; }; For the purposes of a range-based for loop, this is an iterator because it looks like an iterator.
The value is calculated in the do_fib() function:
constexpr void do_fib() { const T prev_n2 = n2; n2 += n1; n1 = prev_n2; } This simply adds n1 + n2 , stores the result in n2 , and stores the old n2 in n1 , setting it up for the next iteration.
The dereference operator * returns the value from n2 , which is the next value in the sequence:
T operator*() const { return n2; } The end() function creates a sentinel fib_generator object, where the count variable is equal to the stop variable:
const fib_generator end() const { auto sentinel = fib_generator(); sentinel.count = stop; return sentinel; } Now the not-equal comparison operator != can easily detect the end of the sequence:
bool operator!=(const fib_generator& o) const { return count != o.count; }
If we want to make our generator work with the algorithm library, we need to provide the traits aliases. These go at the top of the public scope:
public: using iterator_concept = std::forward_iterator_tag; using iterator_category = std::forward_iterator_tag; using value_type = std::remove_cv_t<fib_t>; using difference_type = std::ptrdiff_t; using pointer = const fib_t*; using reference = const fib_t&; Now we can use our generator with algorithms:
fib_generator fib(10); auto x = fib | ranges::views::transform( [](unsigned long x){ return x * x; }); printc(x, "squared:"); This uses the ranges::views version of the transform() algorithm to square each value from the generator. The resulting object can be used wherever you would use an iterator. We get this output from the printc() call:
squared:: 1 1 4 9 25 64 169 441 1156 3025 Coroutines are an asynchronous concurrency paradigm. Introduced in C++20, coroutines allow functions to pause and resume, providing a mechanism for lazy evaluation . Simply stated, lazy evaluation means that a calculation is not performed until required. This makes coroutines well-suited for implementing generators, especially when significant complexity is involved in generating their values.
C++23 provides a generator class that uses the coroutine library. Using the generator class, a simple generator may look like this:
std::generator<long> gen_seq(long count) { for (long i {}; i < count; ++i) { co_yield i; } } int main() { for (auto i : gen_seq(10)) { print("{} ", i); } print("\n"); } Output:
0 1 2 3 4 5 6 7 8 9 The std::generator class is defined in the <generator> header. The co_yield statement, introduced in C++20, makes the gen_seq() function a coroutine . Upon reaching the co_yield statement, the gen_seq() function returns a std::generator object to the caller with the generated value. A coroutine function must have a valid coroutine class as its return type. The co_yield statement suspends execution of the coroutine function to allow lazy evaluation of its generated values. When called again, the coroutine function continues where it left off.
In this recipe, we will create our own generator coroutine class using the coroutine library. While it often makes sense to simply use the provided generator library class, you may find it useful to understand how it implements the coroutine specification to make it work.
This recipe is more complex than the iterator generator in this chapter. It is also more complex than using the C++23 generator STL class. The value is in learning how coroutines and iterators can work together to create effective solutions to common problems.
We'll cover more details about coroutines in , Concurrency and Parallelism .
As of mid-2026, the std::generator class is not yet implemented on Apple Clang. This code has been tested on GCC 15.2 and Microsoft MSVC 1944.
In this recipe, we'll create a simple coroutine generator class that works like the std::generator class provided in the C++23 STL. This will give us some insight into how to implement an iterator with a coroutine.
Coroutines are a relatively new concept in C++ and the concept may seem a bit abstract at first. There are a number of concepts involved with which you may not be familiar, especially if you're new to asynchronous programming. You may want to take your time with this or simply use the std::generator class if that's available to you.
co_gen class. This is a template class, and it begins like any coroutine class: template<typename T> class co_gen { public: struct promise_type; // forward declaration using handle_type = std::coroutine_handle<promise_type>; handle_type coro {}; Our class is called co_gen , for coroutine generator . This class has another class, promise_type , nested within it. The promise_type class requires a forward declaration so it can be used as a template parameter for the coroutine handle, which is named coro .
coroutine_handle template class manages the coroutine itself. We use handle_type as an alias for coroutine_handle with promise_type as its template parameter. The constructor takes a handle_type parameter and initializes coro with the handle of the coroutine: co_gen(handle_type h) : coro(h) {} public interface of our class: ~co_gen() { if (coro) coro.destroy(); } operator bool() { coro.resume(); return !coro.done(); } T operator()() { return coro.promise().value; } The destructor calls coro.destroy() , which destroys the coroutine.
The bool() operator calls the coroutine resume() function, so the generator may produce a new value. It returns the negated done() value from the coroutine_handle object to indicate completion of the coroutine. This allows us to check the status of the coroutine conveniently from a while loop.
The function call operator, operator()() , returns the most recent value from the coroutine via the promise object.
promise_type object for managing values and state The promise_type class is nested within the co_gen class. This provides the interface between the asynchronous coroutine and its caller:
struct promise_type { T value; co_gen get_return_object() { return {handle_type::from_promise(*this)}; } std::suspend_always initial_suspend() { return {}; } std::suspend_always final_suspend() noexcept { return {}; } void return_void() {} std::suspend_always yield_value(T v) { value = v; return {}; } void unhandled_exception() { std::exit(1); } }; The promise_type class is defined as a struct because it has only public scope member functions.
The get_return_object() method returns a pointer to the promise object.
The initial_suspend() and final_suspend() methods return a suspend_always object, to indicate that this is a lazy execution coroutine, as opposed to a greedy (strict evaluation) coroutine.
The return_void() and unhandled_exception() methods are required for compliance with the coroutine standard.
The yield_value() method stores the value in the promise object and returns a suspend_always object.
co_gen class, we can use it from a coroutine function, just as we would with the std::generator class: template<typename T> co_gen<T> gen_seq(T count) { for (T i {}; i < count; ++i) { co_yield i; } } The coroutine return type is a co_gen object, which allows us to invoke the coroutine from main() like this:
int main() { using ulong = unsigned long; auto gen = gen_seq<ulong>(10); while (gen) { print("{} ", gen()); } print("\n"); } This gives us the output:
0 1 2 3 4 5 6 7 8 9 A coroutine is simply a concurrency paradigm that allows lazy evaluation in C++. The coroutine_handle type provides a hook to the suspended or executing coroutine, and the promise_type stores state and value to interface with the caller.
The co_gen class provides the hooks for the coroutine, but the coroutine itself is in the gen_seq() function:
template<typename T> co_gen<T> gen_seq(T count) { for (T i {}; i < count; ++i) { co_yield i; } } The return type of the gen_seq() function is the co_gen class, but the function does not have a return statement. The co_yield statement indicates that the function is a coroutine and returns each value in sequence to the caller. The promise_type class is defined within the co_gen class to provide the necessary hooks into the coroutine system:
struct promise_type { T value; suspend_always initial_suspend() { return {}; } suspend_always final_suspend() noexcept { return {}; } void return_void() {} void unhandled_exception() { exit(1); } co_gen get_return_object() { return {handle_type::from_promise(*this)}; } std::suspend_always yield_value(T v) { value = v; return {}; } }; We've implemented the promise methods necessary for the coroutine:
get_return_object() returns a pointer to the promise object initial_suspend() and final_suspend() return a suspend_always object return_void() and unhandled_exception() are required for compliance with the coroutine standard yield_value() stores the value in the promise object and returns a suspend_always object This provides the basic operation of the coroutine generator. But it still doesn't operate as an iterator. We can now add iterator functionality to our generator.
In order to make the generator work with ranges , views , and range-based for loops, like the C++23 generator class, we need to add an iterator.
The iterator class is defined within the co_gen class. This makes the generator work as an iterable object, just as it would for a container:
class iterator { handle_type coro {}; public: using iterator_concept = std::forward_iterator_tag; using iterator_category = std::forward_iterator_tag; using value_type = std::remove_cv_t<T>; using difference_type = std::ptrdiff_t; using pointer = T*; usingreference=T&; iterator() : coro(nullptr) {} iterator(handle_type h) : coro(h) {} T& operator*() const { return coro.promise().value; } T* operator->() const { return&coro.promise().value; } iterator& operator++() { coro.resume(); if (coro.done()) coro = nullptr; return *this; } iterator operator++(int) { iterator temp = *this; ++*this; return temp; } bool operator==(const iterator& other) const { return coro == other.coro; } bool operator!=(const iterator& other) const { return !(*this == other); } }; iterator begin() { if (coro) coro.resume(); if (coro.done()) return iterator{nullptr}; return iterator{coro}; } iterator end() { return iterator{nullptr}; } Most of the iterator class looks exactly like we would expect of a container iterator. The main differences are in our interface between the coroutine_handle and the promise object.
A few of the iterator methods are of particular interest:
coro object is a handle to the coroutine. We use this to access the promise object which contains the state and data for the coroutine. operator*() returns the value from the promise object. operator++() calls coro.resume() to resume a suspended coroutine. begin() and end() methods are defined outside the scope of the iterator class, so they may operate on the generator class itself. The begin() method calls coro.resume() to resume a suspended coroutine. We can now use our co_gen class just as we would the C++23 generator class. Here's a Fibonacci generator:
template<typename T> co_gen<T> gen_fib(T count) { T n1 {}, n2 {1}; auto do_fib = [&n1, &n2] { const T prev_n2 = n2; n2 += n1; n1 = prev_n2; return n1; }; for (T i = 0; i < count; ++i) { co_yield do_fib(); } } The Fibonacci calculation is encapsulated in a lam b da expression for convenience. We can call this coroutine generator from main() as we would any range or container:
for (auto i : gen_fib<ulong>(10)) { print("{} ", i); } print("\n"); Output:
1 1 2 3 5 8 13 21 34 55 We can even pipe it into standard ranges and views:
auto gen2 = gen_fib<ulong>(10); for (auto i : gen2 | views::transform([](ulong x){ return x * x; })) { print("{} ", i); } print("\n"); This provides us with all the functionality of the C++23 generator class.
Many scripting languages include a function for zipping two sequences together. A typical zip operation will take two input sequences and return a pair of values for each position in both inputs.
Consider the case of two sequences – they may be containers, iterators, or initialization lists:
Figure 4.6: Containers to be zipped
We want to zip them together to make a new sequence with pairs of elements from the first two sequences:
Figure 4.7: Zip operation
In this recipe we will use many of the techniques discussed in this chapter to build a zip iterator with an iterator adaptor.
In this recipe we'll build a zip iterator adaptor that takes two containers of the same type and zips the values into std::pair objects:
main() function we want to call our adaptor with two vectors: int main() { vector<std::string> vec_a {"Bob", "John", "Joni"}; vector<std::string> vec_b {"Dylan", "Williams", "Mitchell"}; print("zipped: "); for(const auto& [a, b] : zip_iterator(vec_a, vec_b)) { print("[{}, {}] ", a, b); } print("\n"); } This allows us to use the zip_iterator in place of the individual vector iterators.
And we expect an output like this:
zipped: [Bob, Dylan] [John, Williams] [Joni, Mitchell] zip_iterator . We'll start with some type aliases for convenience: template<typename T> class zip_iterator { using val_t = typename T::value_type; using ret_t = std::pair<val_t, val_t>; using it_t = typename T::iterator; These allow us to conveniently define objects and functions.
begin() and end() iterators: it_t ita {}; it_t itb {}; it_t ita_begin {}; it_t itb_begin {}; it_t ita_end {}; it_t itb_end {}; ita and itb are iterators from the target containers. The other four iterators are used to generate the begin() and end() iterators for the zip_iterator adaptor.
zip_iterator(it_t ita, it_t itb) : ita {ita}, itb {itb} {} This is used later to construct adaptor objects, specifically for begin() and end() iterators.
public section, we start with the iterator traits type definitions: public: using iterator_concept = std::forward_iterator_tag; using iterator_category = std::forward_iterator_tag; using value_type = std::pair<val_t, val_t>; using difference_type = long int; using pointer = const val_t*; using reference = const val_t&; zip_iterator(T& a, T& b) : ita {a.begin()}, itb {b.begin()}, ita_begin {ita}, itb_begin {itb}, ita_end {a.end()}, itb_end {b.end()} {} zip_iterator& operator++() { ++ita; ++itb; return *this; } bool operator==(const zip_iterator& o) const { return ita == o.ita || itb == o.itb; } bool operator!=(const zip_iterator& o) const { return !operator==(o); } ret_t operator*() const { return {*ita, *itb}; } begin() and end() functions return the respective iterators: zip_iterator begin() const { return zip_iterator(ita_begin, itb_begin); } zip_iterator end() const { return zip_iterator(ita_end, itb_end); } These are made simple by the stored iterators and the private constructor.
main() function for testing: int main() { vector<std::string> vec_a {"Bob", "John", "Joni"}; vector<std::string> vec_b {"Dylan", "Williams", "Mitchell"}; print("vec_a: "); for(const auto& e : vec_a) print("{} ", e); print("\n"); print("vec_b: "); for(const auto& e : vec_b) print("{} ", e); print("\n"); print("zipped: "); for(const auto& [a, b] : zip_iterator(vec_a, vec_b)) { print("[{}, {}] ", a, b); } print("\n"); } vec_a: Bob John Joni vec_b: Dylan Williams Mitchell zipped: [Bob, Dylan] [John, Williams] [Joni, Mitchell] The zipped iterator adaptor is an example of how flexible the iterator abstraction can be. We take the iterators of two containers and use them in one aggregated iterator. Let's see how this works.
The main constructor for the zip_iterator class takes two container objects. For the purposes of this discussion, we'll refer to these objects as the target objects.
zip_iterator(T& a, T& b) : ita {a.begin()}, itb {b.begin()}, ita_begin {ita}, itb_begin {itb}, ita_end {a.end()}, itb_end {b.end()} {} The constructor initializes the ita and itb variables from the target begin() iterators. These will be used to navigate the target objects. The target begin() and end() iterators are also saved for later use.
These variables are defined in the private section:
it_t ita {}; it_t itb {}; // for begin() and end() objects it_t ita_begin {}; it_t itb_begin {}; it_t ita_end {}; it_t itb_end {}; The it_t type is defined as the type of the target iterator class:
using val_t = typename T::value_type; using ret_t = std::pair<val_t, val_t>; using it_t = typename T::iterator; The other aliased types are val_t for the type of the target value and ret_t for the return pair . These type definitions are used for convenience throughout the class.
The begin() and end() functions use a private constructor that only initializes the ita and itb values:
zip_iterator begin() const { return zip_iterator(ita_begin, itb_begin); } zip_iterator end() const { return zip_iterator(ita_end, itb_end); } The private constructor looks like this:
zip_iterator(it_t ita, it_t itb) : ita {ita}, itb {itb} {} This is a constructor that takes it_t iterators for parameters. It only initializes ita and itb so they can be used in the comparison operator overloads.
The rest of the class just acts like a normal iterator, but it operates on iterators from the target class:
zip_iterator& operator++() { ++ita; ++itb; return *this; } bool operator==(const zip_iterator& o) const { return ita == o.ita || itb == o.itb; } bool operator!=(const zip_iterator& o) const { return !operator==(o); } The dereference operator returns a std::pair object ( ret_t is an alias for std::pair<val_t, val_t> ). This is the interface for retrieving a value from the iterator.
ret_t operator*() const { return {*ita, *itb}; } The zip_iterator adaptor can be used to easily zip objects into a map :
map<string, string> name_map{}; for(const auto& [a, b] : zip_iterator(vec_a, vec_b)) { name_map.try_emplace(a, b); } print("name_map: "); for(const auto& [a, b] : name_map) { print("[{}, {}] ", a, b); } print("\n"); When we add this code to main() , we get this output:
name_map: [Bob, Dylan] [John, Williams] [Joni, Mitchell] This recipe is an example of a full-featured contiguous/random-access iterator. This is the most complete type of iterator for a container. A random-access iterator includes all the features of the other types of container iterators, along with its random-access capabilities.
While I felt it was important to include a complete iterator in this chapter, with over 200 lines of code this example is somewhat larger than the other examples in this book. I'll cover the essential components of the code here.
You may download the full source code at .
To start, we need a container for our iterator. We'll use a simple array for this, and we'll call our class, Container . The iterator class is nested within the Container class.
All of this is designed to be consistent with the STL container interfaces.
Container is defined as a template class. Its private section has only two elements: template<typename T> class Container { std::unique_ptr<T[]> c_ptr {}; size_t n_elements{}; We use a unique_pointer for the data. We let the smart pointer manage its own memory. This mitigates the need for a ~Container() destructor. The n_elements variable keeps the size of our container.
For a discussion of smart poniters , see C hapter 8 , Utility Classes .
explicit Container(size_t sz) : n_elements{sz} { c_ptr = std::make_unique<T[]>(n_elements); } The make_unique() call constructs empty objects for element.
Container(initializer_list<T> i_lst) : Container(i_lst.size()) { std::copy(i_lst.begin(), i_lst.end(), c_ptr.get()); } size() method returns the number of elements: size_t size() const { return n_elements; } operator[]() overload returns an indexed element without bounds checking : const T& operator[](const size_t index) const { return c_ptr[index]; } at() method returns an indexed element with bounds checking : T& at(const size_t index) const { if (index > n_elements - 1) { throw std::out_of_range( "Container::at(): index out of range" ); } return c_ptr[index]; } This is consistent with STL usage. The at() function is the preferred method of addressing an indexed element.
begin() and end() methods call the iterator constructor with the address of the container data iterator begin() const { return iterator(c_ptr.get()); } iterator end() const { return iterator(c_ptr.get() + n_elements); } The unique_ptr::get() function returns the address from the smart pointer.
iterator class is nested within the Container class as a public member class iterator { T* ptr; The iterator class has one private member, a pointer that's initialized in the begin() and end() methods of the Container class.
iterator(T* ptr = nullptr) : ptr{ptr} {} We provide a default value because the standard requires a default constructor.
This iterator provides operator overloads for the following operators: ++ , postfix ++ , ‑‑ , postfix ‑‑ , [] , default comparison <=> (C++20) , == , * , ‑> , + , non-member + , numeric ‑ , object ‑ , += , and ‑= . We'll cover a few notable overloads here. See the source code for all of them.
<=> provides the functionality of the full suite of comparison operators, except the equality == operator: // default comparison operator (C++20) const auto operator<=>(const iterator& o) const { return ptr <=> o.ptr; } This is a C++20 feature, so it requires a compliant compiler and library.
+ operator overloads. These support it + n and n + it operations. iterator operator+(const size_t n) const { return iterator(ptr + n); } // non-member operator (n + it) friend const iterator operator+(const size_t n, const iterator& o) { return iterator(o.ptr + n); } The friend declaration is a special case. When used in a template class member function, it's the equivalent of a non-member function. This allows a non-member function to be defined in the class context.
‑ operator also has two overloads. We need to support both a numeric operand and an iterator operand. iterator operator-(const size_t n) { return iterator(ptr - n); } size_t operator-(const iterator& o) { return ptr - o.ptr; } This allows both it – n and it – it operations. There's no need for a non-member function, as n – it is not a valid operation.
The C++23 specification requires a specific set of operations and results for a valid random-access iterator. I've included a unit_tests() function in the source code to validate those requirements.
The main() function creates a Container object and performs some simple validation functions.
Container<string> object x with ten values Container<string> x{"one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten" }; println("Container x size: {}", x.size()); The output gives the number of elements:
Container x size: 10 println("Container x:"); for (auto e : x) { print("{} ", e); } print("\n"); Output:
Container x: one two three four five six seven eight nine ten println("direct access elements:"); println("element at(5): {}", x.at(5)); println("element [5]: {}", x[5]); println("element begin + 5: {}", *(x.begin() + 5)); println("element 5 + begin: {}", *(5 + x.begin())); println("element begin += 5: {}", *(x.begin() += 5)); Output:
direct access elements: element at(5): six element [5]: six element begin + 5: six element 5 + begin: six element begin += 5: six ranges::views pipe and views::reverse : println("views pipe reverse:"); auto result = x | views::reverse; for (auto v : result) print("{} ", v); print("\n"); Output:
views pipe reverse: ten nine eight seven six five four three two one Container object with 10 uninitialized elements: Container<string> y(x.size()); println("Container y size: {}\n", y.size()); for (auto e : y) { print("[{}] ", e); } print("\n"); Output:
Container y size: 10 [] [] [] [] [] [] [] [] [] [] unit_tests() function runs the tests from the standard and provides the following output: unit tests ============================= *a: one *b: six n: 5 true: (a += n) == b true: std::addressof(a += n) == std::addressof(a) true: (a + n) == (a += n) true: (a + n) == (n + a) true: a + (i + j) == (a + i) + j true: a + 0 == a true: --b == (a + (n - 1)) true: (b += -n) == a && (b -= n) == a true: std::addressof(b -= n) == std::addressof(b) true: (b - n) == (b -= n) true: a[n] == *b true: (a <= b) Although it's a lot of code, this iterator is no more complicated than a smaller iterator. Most of the code is in the operator overloads, which are mostly just one or two lines of code each.
The data is managed by a smart pointer , which is simplified by the fact that it's a flat array and doesn't require expansion or compression. We'll cover smart pointers in more detail in C hapter 8 , Utility Classes .
While the STL provides a flat std::array class, as well as other more complex data structures, you will find it valuable to demystify the workings of a complete iterator class.
Scan the QR code (or go to ). Search for this book by name, confirm the edition, and then follow the steps on the page.
Note: Keep your invoice handy. Purchases made directly from Packt don't require an invoice.