7

Strings and Formatting

The STL string class is a powerful, full-featured tool for storing, manipulating, and displaying character-based data. It has much of the convenience you would find in a high-level scripting language while providing the agility and efficiency you expect from C++.

The string class is based on basic_string , a contiguous container class that may be instantiated with any character type. Its template class signature looks like this:

template<     typename CharT,     typename Traits = std::char_traits<CharT>,     typename Allocator = std::allocator<CharT> > class basic_string;

The Traits and Allocator template parameters are usually left to their default values.

The underlying storage of basic_string is a contiguous sequence of CharT , and can be accessed with the data() member function:

const std::basic_string<char> s {"hello"}; const char * sdata = s.data(); for(size_t i{}; i < s.size(); ++i) {     print("{} ", sdata[i]); } print("\n");

Output:

h e l l o

The data() member function returns a CharT* that points to the underlying array of characters. Since C++11, the array returned by data() is null-terminated, making data() equivalent to c_str() .

The basic_string class includes many of the methods you would find in other contiguous-storage classes, including insert() , erase() , push_back() , pop_back() , and others. These methods operate on the underlying array of CharT .

std::string is a type alias for std::basic_string<char> :

using std::string = std::basic_string<char>;

For most purposes, you'll use std::string .

Before C++20, string formatting had been a weak point in the STL. We were given the imperfect choice between the cumbersome STL iostreams or the archaic legacy printf() . Beginning with C++20 and the format library, STL string formatting has finally grown up. Closely based on Python's str.format() method, the new format library is fast and flexible, providing many of the advantages of both iostreams and printf() , along with good memory management and type safety.

In this chapter, we will cover these subjects and more in the following recipes:

• Use string_view as a lightweight string object
• Format text with the format library
• Print strings with std::print and std::println
• Concatenate strings
• Transform strings
• Trim whitespace from strings
• Read strings from user input
• Count words in a file
• Initialize complex structures from file input
• Customize a string class with char_traits
• Parse strings with regular expressions

Use string_view as a lightweight string object

The string_view class provides a lightweight alternative to the string class. Instead of maintaining its own data store, string_view operates on a view of a C-string. This makes string_view smaller and more efficient than std::string . It's useful in cases where you need a string object but don't need the more memory- and computation-intensive features of std::string .

For example, we've been using a function like this in many of our recipes:

void printc(const auto& c, string_view s = "") {     if (s.size()) print("{}: ", s);     for (const auto& e : c) print("{} ", e);     print("\n"); }

This function prints the contents of a container, along with an optional description string, like this output from the permutations example in C hapter 6 :

dest: cat dog kirk spock sulu velociraptor

We pass the description string to the printc() function as a literal C-string:

printc(dest, "dest");

The literal string is used to construct the string_view object s in printc() . string_view is convenient here because we're able to concisely check the size() method to see if there is a description to print:

if (s.size()) print("{}: ", s);

We could have instead used a reference to a std::string :

void printc(const auto& c, const string& s = "")

This would construct a string object from the literal c-string.

Because the string object keeps its own copy of the data it must first be copied, which uses both space and compute cycles. On the other hand, string_view simply refers to the original C-string for its data, thus providing much of the utility of std::string without the additional cost of copying and storing the data.

The underlying data is available via the string_view::data() method, which returns a const -qualified pointer.

How to do it

The string_view class looks deceptively similar to the STL string class, but it works a bit differently. Let's consider some examples:

• Here's an STL string initialized from a C-string (array of char ):

  char text[] {"hello"}; string greeting {text}; text[0] = 'J'; println("{} {}", text, greeting);

  Output:

  Jello hello

  Notice that we modify the text array with text[0] = 'J' . This does not change the characters in the original char array. This is because string object carries its own copy of the underlying data.

• When we do the same with a string_view , we get a different result:

  char text[] {"hello"}; string_view greeting {text}; text[0] = 'J'; println("{} {}", text, greeting);

  Output:

  Jello Jello

  The string_view object creates a view of the underlying data. It does not make its own copy. This results in significant efficiencies but also allows for side effects.

• Because string_view doesn't copy the underlying data, the source data must remain in scope for the duration of the string_view object. So, this does not work:

  string_view sv() {     const char text[] {"hello"};  // temporary storage     string_view greeting {text};     return greeting; }  int main() {     auto greeting = sv();     // data out of scope     println("{}", greeting);  // output undefined }

  Because the underlying data is out of scope in the main() function, the greeting object in main() is no longer valid by the time we use it, resulting in undefined output and possible memory leaks.

• The string_view class has constructors that work for different types of underlying containers. This includes character arrays ( const char* ), contiguous ranges (including std::string ), and other string_view objects. For example, constructing a string_view from a string uses a ranges constructor:

  string str {"hello"}; string_view greeting {str}; println("{}", greeting);

  Output:

  hello

• There is also a string_view literal operator sv , defined in the std::literals namespace:

  using namespace std::literals; auto sv = "hello"sv; println("{}", sv.substr(1,4));

  This constructs a constexpr string_view object and calls its method substr() to return the 4 values starting at index 1 (the second character).

  Output:

  ello

How it works

The string_view class is effectively an iterator adaptor on a contiguous sequence of characters. The implementation typically has two members: a const CharT* and a size_t . It works by wrapping a contiguous_iterator around the source data.

This means that you can use it like std::string for many purposes, with a few important distinctions:

• The copy constructor does not copy the data. This means that when you make a copy of a string_view , each copy operates on the same underlying data:

  char text[] {"hello"}; string_view sv1 {text}; string_view sv2 {sv1}; string_view sv3 {sv2}; string_view sv4 {sv3}; println("{} {} {} {}", sv1, sv2, sv3, sv4); text[0] = 'J'; println("{} {} {} {}", sv1, sv2, sv3, sv4);

  Output:

  hello hello hello hello Jello Jello Jello Jello

• Keep in mind that when you pass a string_view to a function, it uses the copy constructor:

  void f(string_view sv) {     if (sv.size()) {         char* x = (char*) sv.data();  // dangerous         x[0] = 'J';  // modifies the source     }     println("f(sv): {} {}", (void*) sv.data(), sv); }  int main() {     char text[] {"hello"};     string_view sv1 {text};     println("sv1: {} {}", (void*) sv1.data(), sv1);     f(sv1);     println("sv1: {} {}", (void*) sv1.data(), sv1); }

  Output:

  sv1: 0x7fff595bcb0a hello f(sv): 0x7fff595bcb0a Jello sv1: 0x7fff595bcb0a Jello

  Notice that the address of the underlying data (returned by the data() member function) is the same for all instances of the string_view . That's because the string_view class does not make a copy of the underlying data. Even though the string_view member pointer is const -qualified, it's still possible to cast away the const qualifier, though that's not recommended because it could cause unintended side effects. But it is worth noting that the data is never copied.

• Because the string_view class does not own its source data, it lacks methods that directly operate on the underlying string. Methods such as append() , operator+() , push_back() , pop_back() , replace() , and resize() , which are supported in string , are not supported in string_view .

If you need to concatenate strings with the + operator, you'll need a std::string object. For example, this does not work with string_view :

sv1 = sv2 + sv3 + sv4; // does not work

You'll need to use string instead:

char text[] {"hello"}; string str1 {text}; string str2 {str1}; string str3 {str2}; string str4 {str3}; str1 = str2 + str3 + str4; // works println("{}", str1);

Output:

hellohellohello

We may access the underlying data of a string_view object with its data() method.

auto str_data = sv1.data(); for (auto i = 0; i < sv1.size(); ++i) { print("{} ".format(str_data[i])) } print("\n")

Output:

J e l l o

Format text with the format library

Before C++20, if you wanted to format text, you could use either the legacy printf functions or the STL iostream library. Both have their strengths and flaws.

The printf -based functions are inherited from C and have proven efficient, flexible, and convenient for over 50 years. The formatting syntax can look a bit cryptic, but it's simple enough once you get used to it.

printf("Hello, %s\n", c_string);

The main weakness in printf is its lack of type safety. The common printf() function (and its relatives) use C's variadic arguments model to pass parameters to a formatter. This works great when it works, but it can cause serious problems when a parameter type doesn't match its corresponding format specifier. Modern compilers do as much type-checking as they can, but the model is inherently flawed and the protection can only go so far.

The STL iostream library brings type safety at the expense of readability and run-time performance. The iostream syntax is unusual, yet familiar. It overloads the bitwise left-shift operator ( << ) to allow a chain of objects, operands, and formatting manipulators , which produce the formatted output.

cout << "Hello, " << str << endl;

The weakness of iostream is its complexity, in both syntax, and implementation. Building a formatted string can be verbose and obscure. Many of the formatting manipulators must be reset after use, or they create cascading formatting errors that can be difficult to debug. The library itself is vast and complex, resulting in code significantly larger and slower than its printf equivalent.

This left C++ programmers with little option but to choose between two flawed systems. The new format library, introduced with C++20, provides an elegant and efficient solution to this dilemma.

The format library is in the <format> header. As of mid-2024, format is implemented and supported on all the major compiler systems. The format library is modeled on the str.format() method from Python 3. Format strings are substantially the same as those in Python and, for most purposes, they should be interchangeable. Let's examine some simple examples:

How to do it

• In its simplest form, the format() function takes a string_view format string and a variadic parameter pack of arguments. It returns a string . Its function signature looks like this:

  template<typename... Args> string format(string_view fmt, const Args&...args);

• The format() function returns a string with a textual representation of virtually any type or value. For example:

  string who {"everyone"}; int ival {42}; double pi {std::numbers::pi}; auto s1 = format("Hello, {}!", who); auto s2 = format("Integer: {}", ival); auto s3 = format("π: {}", pi);  for (auto s : {s1, s2, s3}) {     println("{}", s); }

  Output:

  Hello, everyone! Integer: 42 π: 3.141592653589793

  The format string uses braces {} as a placeholder. With no format specifiers, the braces are effectively a type-safe placeholder which will convert a value of any compatible type to a reasonable string representation.

• You can include multiple placeholders in your format string, like this:

  auto s4 = format("Hello {} {}", ival, who);

  Output:

  Hello 42 everyone

• You can specify the order of the replacement values. This could be useful for internationalization:

  auto s5 = format("Hello {1} {0}", ival, who); auto s6 = format("Hola {0} {1}", ival, who);

  Output:

  Hello everyone 42 Hola 42 everyone

• You can align values, left ( < ), right ( > ), or center ( ^ ), with or without a fill character:

  auto s7 = format("{:.<10}", ival); auto s8 = format("{:.>10}", ival); auto s9 = format("{:.^10}", ival);

  Output:

  42........ ........42 ....42....

• You can set the decimal precision of values:

  auto s10 = format("π: {:.5}", pi);  // π: 3.1416

  And much, much more.

  It's a rich and complete formatting specification that provides the type-safety of iostream with the performance and simplicity of printf , for the best of both worlds.

How it works

The format library is the basis of the print() function, which was introduced with C++23. The format() function itself returns a string object.

The format library uses a few separate functions to do its work. By way of example, we could create our own print() function using components from the format library:

template<typename... Args> void print(const string_view fmt_str, Args&&... args) {     auto fmt_args = std::make_format_args(args...);     auto outstr = std::vformat(fmt_str, fmt_args);     fputs(outstr.c_str(), stdout); }

This uses the same arguments as the format() function, and it demonstrates the use of make_format_args() and vformat() to format output. The first argument is a string_view object for the format string. This is followed by a variadic parameter pack for the arguments.

The make_format_args() function takes the parameter pack and returns an object with type-erased values suitable for formatting. This object is then passed to vformat() , which returns a string suitable for printing. We use fputs() from the C standard library to print the formatted string to the console, but you could just as easily store the string in a database or use it for any purpose.

In C++, type-erased values are objects that hide their concrete type behind a uniform interface. This allows a class like std::format to operate on values of different types without knowing their types at compile time.

There's more…

It's nice to have the ability to format strings and primitives, but for the format library to be fully functional, it needs customization to work with your own classes.

For example, here's a simple struct with two members: a numerator and denominator . We'd like this to print as a fraction:

struct Frac {     long n;     long d; }; int main() {     Frac f {5, 3};     print("Frac: {}\n", f); }

When I compile this, it leads to a cascade of errors to the effect of, "No user-defined conversion operator…". Okay, so, let's fix it!

When the format system encounters an object for conversion , it looks for a specialization of a formatter object with the corresponding type. Standard specializations are included for common objects such as strings and numbers.

It's quite simple to create a specialization for our Frac type:

template<typename T> struct Frac {     T n;     T d; };  template <typename T> struct std::formatter<Frac<T>>: std::formatter<int> {     template<typename FormatContext>     auto format(const Frac<T>& o, FormatContext& ctx) const {         return format_to(ctx.out(), "{}/{}", o.n, o.d);     } };

The format() function is an overload of the std::format() function that takes an object of the type we want to specialize for, in this case a Frac object, and a FormatContext object which is inherited from the formatter . This version of format() returns an end iterator . The format_to() function takes that iterator, the format string, and the variadic parameters. In this case, the variadic parameters are the two values from our Frac class, the numerator and denominator.

All we need to do is provide a simple format string "{0}/{1}" and the numerator and denominator values. The 0 and 1 indicate the position of the parameters. This is not strictly necessary but they could come in handy in other contexts.

Now that we have a specialization for Frac , we can pass our object to print() to get a readable result:

int main() {     Frac f {5, 3};     print("Frac f {{5, 3}}: {}\n", f); }

Output:

Frac f {5, 3}: 5/3

Note the double braces {{ and }} in the format string. You can use double braces to print a single brace without having it interpreted as a formatting parameter by the library.

The C++20 format library solves a long-standing problem by providing a type-safe text formatting library that is both efficient and convenient.

See also…

A good reference for format string options is available at:

Print strings with std::print and std::println

While cout has had its place since the beginning of the STL, it has been outdated for many years. The advent of type-safe variadic templates and parameter packs with C++11 made it possible to replace cout with a more succinct and efficient alternative, like printf() but with type safety.

New for C++23, the print() and println() functions leverage the format library to provide efficient string formatting and printing for both console and file streams. This was originally proposed for C++20 but was not ready in time for ratification.

Because the print() functions bypass the cumbersome iostream library, these functions are far more efficient than using format with cout . Performance and code size are comparable to printf() but with the type-safety benefits of the format library.

The print() and println() functions are defined in the <print> header.

How to do it

The print() and println() functions are simple and powerful.

• To use the print() and println() functions, we need to include the <print> header:

  #include <print>

• I also like to set up a using declaration to allow use of the print and println symbols without the std:: namespace:

  using std::print, std::println;

  It's often considered best practice to keep to one symbol per using declaration, but I make an exception when the symbols are so closely related.

• Now we can more easily use these functions in our code:

  int main() {     const char* greeting {"earthlings"};     print("hello, {}!\n", greeting); }

  Our output:

  hello, earthlings!

• The println() function works just like print() but it sends a newline after printing the string:

  print("one"); print("two"); print("three"); print("\n"); println("one"); println("two"); println("three");

  Output:

  onetwothree one two three

  The println() variant is useful in many circumstances where you want to print a line of text at a time.

• Beginning with C++23, we can also use these functions to write to a file stream:

  const char* greeting {"earthlings"}; const char* filename {"output.txt"}; std::ofstream ofile(filename); if (ofile.is_open()) {     println("Writing to {}", filename);     println(ofile, "hello, {}!", greeting);     ofile.close();     println("done."); } else {     println(stderr, "failed to open file {}", filename); }

  Output:

  Writing to output.txt done.

  Contents of output.txt :

  hello, earthlings!

  This makes printing to a file just as simple as printing to the console.

How it works

The print() and println() functions leverage the formatter class to expand variables and positional arguments to format the output according to the options specified in a format string . The function signature is effectively the same as that of format() , with the distinction being only the return type:

template< class... Args > std::string format(std::format_string<Args...> fmt, Args&&...args);  template< class... Args > void print(std::format_string<Args...> fmt, Args&&...args);  template< class... Args > void println(std::format_string<Args...> fmt, Args&&...args);

This means that you may use print() and println() in the same context as format() , eliminating the need for cout in most, if not all, circumstances. In other words, where you may have used cout with format:

cout << format("hello, {}", greeting) << endl;

You may now just use print() or println() :

println("hello, {}", greeting);

With these functions being part of the standard, you should now favor using print() and println() wherever you would have previously used cout .

There's more…

It's worth mentioning that any specializations for the formatter work seamlessly with print() and println() .

For example, the specialization we used in the format recipe will also work with print() and println() :

template<typename T> struct Frac {     T n;     T d; };  template <typename T> struct std::formatter<Frac<T>>: std::formatter<int> {     template<typename FormatContext>     auto format(const Frac<T>& o, FormatContext& ctx) const {         return format_to(ctx.out(), "{}/{}", o.n, o.d);     } };

This is the same code we used in the format example, and it works just as well with print() and println() :

Frac<int> f {5, 3}; println("Frac f {{5, 3}}: {}", f);

Output:

Frac f {5, 3}: 5/3

This makes it easy to format output for your own classes on the console or in files.

Concatenate strings

There are several ways to concatenate strings in C++. In this recipe, we look at a few of the most common: the string class + operator, the string::append() method, the ostringstream class << operator, and the format() function. Each of these has its advantages, disadvantages, and use cases.

How to do it

In this recipe, we examine ways to concatenate strings. We will then perform some benchmarks and consider the different use cases.

• We'll start with a couple of std::string objects:

  string a {"a"}; string b {"b"};

  The string objects are constructed from literal C-strings.

  The std::string class's C-string constructor makes a copy of the literal string and uses the local copy as the underlying data for the string object.

• Now, let's construct a new empty string object and concatenate a and b with a separator:

  string x {}; x += a + ", " + b; println("{}", x);

  Here, we used the string object's += and + operators to concatenate the a and b strings, along with literal string ", " . The resulting string has the elements concatenated together:

  a, b

• We could instead use the append() method of the string class:

  string x {}; x.append(a); x.append(", "); x.append(b); println("{}", x);

  This gives us the same result:

  a, b

• Or we could construct an ostringstream object, which uses the stream interface:

  ostringstream x {}; x << a << ", " << b; println("{}", x.str());

  Note that we use x.str() to print the ostringstream because format does not have a specialization for iostreams.

  We get the same result:

  a, b

• We could also use the format() function:

  string x {}; x = format("{}, {}", a, b); println("{}", x);

  Again, we have the same result:

  a, b

• Or, because println() uses the format library, we can skip the format step altogether and use println() directly with no intermediate string object:

  println("{}, {}", a, b);

  And we get the same output:

  a, b

How it works

The string object has two distinct methods for concatenating a string: the + operator and the append() member function.

The append() member function adds data to the end of the string object's data. It must allocate and manage memory to accomplish this.

The + operator uses the operator+() overload to construct a new string object with the old and new data and returns the new object.

The ostringstream object works like an ostream but stores its output in a string object. The << operator returns a basic_ostream object.

The C++20 format() function uses a format string with variadic arguments and returns a newly constructed string object.

It's worth noting here that the <print> header also includes the <format> header, so we need only #include <print> which gets us all the code in both headers.

There's more…

How do you decide which concatenation strategy is right for your code? We can start with some benchmarks.

Benchmarks

I performed these tests using GCC 15.2 with ‑O2 optimization on a VM running Debian Linux 6.17. Your results will, of course, be different.

• First, we'll create a timer function using the <chrono> library:

  void timer(string(*f)()) {     auto t1 = high_resolution_clock::now();     string s {f()};     auto t2 = high_resolution_clock::now();     duration<double, std::milli> ms = t2 - t1;     println("{}", s);     println("duration: {} ms", ms.count()); }

  The timer function calls the function passed to it, marking the time before and after the function call. It then displays the duration using cout .

• Now, we create a function that concatenates strings, using the append() member function:

  string append_string() {     println("== append_string:");     string a {"a"};     string b {"b"};     long n {};     while (++n) {         string x {};         x.append(a);         x.append(", ");         x.append(b);         if(n >= 10000000) return x;     }     return "error"; }

  For benchmarking purposes, this function repeats the concatenation 10 million times. We call this function from main() with timer() :

  int main() {     timer(append_string); }

  We get this output:

  == append_string: a, b duration: 74.882582 ms

  So, on this system, our concatenation ran 10 million iterations in about 75 milliseconds.

• Now, let's create the same function with the + operator overload:

  string concat_string() {     println("== concat_string:");     string a {"a"};     string b {"b"};     long n {};     while (++n) {         string x {};         x += a + ", " + b;         if(n >= 10000000) return x;     }     return "error"; }

  Our benchmark output:

  == concat_string: a, b duration: 174.573135 ms

  This version performed 10 million iterations in about 175 milliseconds.

• Now, let's try it with ostringstream :

  string concat_ostringstream() {     println("== ostringstream:");     string a {"a"};     string b {"b"};     long n {};     while (++n) {         ostringstream x {};         x << a << ", " << b;         if(n >= 10000000) return x.str();     }     return "error"; }

  Our benchmark output:

  == ostringstream: a, b duration: 1434.781101 ms

  This version ran 10 million iterations in about 1.4 seconds.

• Finally, here's the format() version:

  string concat_format() {     println("== append_format:");     string a {"a"};     string b {"b"};     long n {};     while (++n) {         string x {};         x = format("{}, {}", a, b);         if(n >= 10000000) return x;     }     return "error"; }

  Our benchmark output:

  == append_format: a, b duration: 929.482062 ms

  The format() version ran 10 million iterations in about 930 milliseconds.

Summary of the results:

Table 7.1: A comparison of concatenation performance

Why the performance discrepancy?

We can see from these benchmarks that the ostringstream version takes many times longer than the string -based versions.

The append() method is fastest. It must allocate memory but does not construct new objects. Some optimizations may be possible due to repetition.

The + operator overload calls the append() method. The additional function call makes it incrementally slower than the append() method.

The ostringstream operator << creates a new ostream object for each operation. With the complexity of the stream object, along with managing the stream state, this makes it much slower than any of the string -based versions.

The format() version creates one new string object but without the overhead of the iostream system.

Why would I choose one over another?

Some measure of personal preference will be involved. The operator overloads ( + or << ) can be convenient. Performance may or may not be an issue for you.

The ostringstream class specializes the << operator for each different type, so it's able to operate in circumstances where you may have different types calling the same code.

The format() function offers the same type-safety and customization options as the ostringstream class, yet is significantly faster.

The string object's + operator is fast, easy to use, and easy to read, but is incrementally slower than append() .

The append() version is fastest but requires a separate function call for each item.

For many purposes, I prefer the format() function or the string object's + operator. I'll use append() if every bit of speed matters.

Transform strings

The std::string class is a contiguous container , much like std::vector or std::array . It supports the contiguous_iterator concept and all corresponding algorithms.

The string class is a specialization of basic_string with a char type. This means that the elements of the container are of type char . Other specializations are available, but string is most common.

Because it is fundamentally a contiguous container of char elements, string may be used with the transform() algorithm, or any other technique that operates on a contiguous_iterator concept.

How to do it

There are several ways to perform transformations, depending on the application. This recipe will explore a few of them.

• We'll start with a few predicate functions. A predicate function takes a transformation element and returns a related element. For example, here is a simple predicate that returns an uppercase character:

  const char char_upper(const char c) {     if (c >= 'a'&&c<= 'z') return c - ('a' - 'A');     else return c; };

  This function depends on the arithmetic relationship between uppercase and lowercase letters in the ASCII code set.

  We could use the std::toupper() function but it returns an int and would require a cast to work with the char elements in a string . When working in ASCII, I prefer to use this simple function instead.

  Here is the corresponding char_lower() function:

  const char char_lower(const char c) {     if (c >= 'A'&&c<= 'Z') return c + ('a' - 'A');     else return c; };

• The rot13() function is a fun transformation predicate for demonstration purposes. It's a simple substitution cypher, not suitable for encryption but commonly used for obfuscation :

  const char rot13s(const char& x) {     auto rot13c =       [](const char x, const char a) -> const char {         return a + (x - a + 13) % 26;     };     if (x >= 'A'&&x<= 'Z') return rot13c(x, 'A');     if (x >= 'a'&&x<= 'z') return rot13c(x, 'a');     return x; }

  This uses a lambda expression to encapsulate the rotation itself. The rotation is accomplished with a simple modulus operation.

• We can use these predicates with the transform() algorithm:

  int main() {     string s {"hello jimi"};     println("{}", s);      ranges::transform(s, s.begin(), char_upper);     println("{}", s);     ...

  I chose the ranges::transform() function for its simplicity. We pass the char_upper() predicate which applies the transformation to each element of s and puts the result back in s :

  Output:

  hello jimi HELLO JIMI

• Instead of transform() , we could use a simple for loop with the predicate function:

  for (auto& c : s) c = rot13(c); println("{}", s);

  Starting with our uppercase string object, the result is:

  URYYB WVZV

• The fun thing about the rot13 cypher is that it unscrambles itself. Because there are 26 letters in the ASCII alphabet, rotating 13 and then rotating 13 again results in the original string. Let's transform to lowercase and rot13 again to restore our string:

  for (auto& c : s) c = rot13(char_lower(c)); println("{}", s);

  Output:

  hello jimi

  Because of their uniform interface, the predicate functions may be chained as parameters of each other. We could also use char_lower(rot13s(c)) with the same result.

• If your requirement is too complex for a simple character-by-character transformation, you may use string iterators as you would with any contiguous container. Here's a simple function that transforms a lowercase string to Title Case by capitalizing the first character and every character that follows a space:

  const string& title_case(string& s) {     auto begin = s.begin();     auto end = s.end();     *begin++ = char_upper(*begin);  // first element     bool space_flag {false};     for (auto it {begin}; it != end; ++it) {         if (*it == ' ') {             space_flag = true;         } else {             if (space_flag) *it = char_upper(*it);             space_flag = false;         }     }     return s; }

  Because it returns a reference to the transformed string, we can call it with print() , like this:

  println("{}", title_case(s));

  Output:

  Hello Jimi

How it works

The std::basic_string class, and its specializations (including string ), are supported by iterators fully compliant with contiguous_iterator . This means that any technique that works with any contiguous container also works with string .

Trim whitespace from strings

It is common for input from users to include extraneous whitespace characters at one or both ends of a string. This can be problematic, so we often need to remove them. In this recipe, we'll use the string class methods, find_first_not_of() and find_last_not_of() , to trim whitespace from the ends of a string.

How to do it

The std::string class includes methods for finding elements that are, or are not, included in a list of characters. We'll use these methods to trim whitespace characters from a string :

• We start by defining string with input from a hypothetical ten-thumbed user:

  int main() {     string s {" \t  ten-thumbed input   \t   \n \t "};     println("[{}]", s);     ...

  Our input has a few extra tab \t and newline \n characters before and after the content. We print it with surrounding brackets to show the whitespace:

  [       ten-thumbed input       ]

• Here's a trimstr() function to remove all the whitespace characters from both ends of a string :

  string trimstr(const string& s) {     constexpr const char * whitespace {" \t\r\n\v\f"};     if (s.empty()) return s;     const auto first = s.find_first_not_of(whitespace);     if (first == string::npos) return {};     const auto last = s.find_last_not_of(whitespace);     return s.substr(first, (last - first + 1)); }

  We define our set of whitespace characters as space , tab , return , newline , vertical tab , and form feed . While some of these are more common than others, that's the canonical set.

  This function uses the find_first_not_of() and find_last_not_of() methods of the string class to find the first and last elements that are not a member of the set.

• Now, we can call the function to get rid of all that unsolicited whitespace:

  s = trimstr(s); println("[{}]", s);

  Output:

  [ten-thumbed input]

How it works

The string class's various find...() member functions return a position as a size_t value:

size_t find_first_not_of( const CharT* s, size_type pos = 0 ); size_t find_last_not_of( const CharT* s, size_type pos = 0 );

The return value is the zero-based position of the first matching character ( not in the s list of characters) or the special value, string::npos , if not found. npos is a static member constant that represents an invalid position.

We test for (first == string::npos) and return an empty string {} if there is no match. Otherwise, we use the first and last positions with the s.substr() method to return the string without whitespace.

Read strings from user input

The STL provides character-based input from the standard input stream using the std::cin object. The cin object is a global singleton that reads input from the console as an istream input stream.

By default, cin reads one word at a time until it reaches the end of the stream:

string word{}; print("Enter words: "); while(cin >> word) {     print("[{}] ", word); } print ("\n");

Output:

$ ./working Enter words: big light in sky [big] [light] [in] [sky]

This is of limited usefulness and it may lead some to dismiss cin as minimally functional. While cin definitely has its quirks, it can be wrangled into providing line-oriented input.

How to do it

To get basic line-oriented functionality from cin , there are two significant behaviors that need to be understood. One is the ability to get a line at a time, instead of a word at a time. The other is the ability to reset the stream after an error condition. Let's look at these in some detail:

• First, we need to prompt the user for input. Here's a simple prompt function:

  bool prompt(const string_view s,   const string_view s2 = "") {     if (s2.size()) print("{} ({}): ", s, s2);     else print("{}: ", s);     fflush(stdout);     return true; }

  The fflush() function is part of the C Standard. This ensures that the output is written to the display or output file immediately. This is necessary because the std::print() function does not guarantee that the output is flushed.

• The cin class has a getline() method that gets a line of text from the input stream and puts it in a C-string array:

  constexpr size_t MAXLINE {1024 * 10}; char s[MAXLINE]{}; const char * p1 {"Words here"}; prompt(p1); cin.getline(s, MAXLINE, '\n'); println("{}", s);

  Output:

  $ ./working Words here: big light in sky big light in sky

  The cin.getline() method takes three arguments:

  getline(char* s, size_t count, char delim );

  The first argument is a C-string buffer for the destination. The second is the size of the C-string buffer. The third is the delimiter for the end of the line.

  The function will not put more than count – 1 characters in the buffer, leaving room for a null terminator .

  The delimiter defaults to the newline '\n' character. I have specified it here for completeness.

• The STL also provides a stand-alone getline() function that works with an STL string object:

  string line{}; const char * p1a{ "More words here" }; prompt(p1a, "p1a"); getline(cin, line, '\n'); println("{}", line);

  Output:

  $ ./working More words here (p1a): slated to appear in east slated to appear in east

  The stand-alone std::getline() function takes three arguments:

  getline(basic_istream&∈,string&str,chardelim);

  The first argument is the input stream. The second is a reference to a string object. And the third is the end-of-line delimiter.

  If not specified, the delimiter defaults to the newline '\n' character.

  I usually find the standalone getline() more convenient than the cin.getline() method.

• We can use cin to get a specific type from the input stream. To do this, we must be able to handle an error condition.

  When cin encounters an error, it sets the stream to an error condition and stops accepting input. To retry input after an error, we must reset the state of the stream. Here's a function that resets the input stream after an error:

  void clearistream() {     int c {};     cin.clear();     while (c != '\n'&&c!=EOF)c=getchar(); }

  The cin.clear() function resets the error flags on the input stream but leaves text in the buffer. We then clear the buffer by reading a line and discarding it. The C Standard getchar() function is the most reliable way to clear the input buffer.

• We can accept numeric input by using cin with numeric type variables:

  double a {}; double b {}; for (prompt(p2); !(cin >> a >> b); prompt(p2)) {     println("not numeric");     clearistream(); } println("You entered {} and {}", a, b); clearistream();

  Output:

  $ ./working Please enter two numbers: a b not numeric Please enter two numbers: 47 73 You entered 47 and 73

  The cin >> a >> b expression accepts input from the console and attempts to convert the first two words to types compatible with a and b ( double ). If it fails, we call clearistream() and try again. We call clearistream() after the loop to ensure the input buffer is cleared.

• We can use the getline() separator parameter to get comma-separated input:

  line = ""; prompt(p3); while(line.empty()) getline(cin, line); stringstream ss(line); while(getline(ss, word, ',')) {     if(word.empty()) continue;     println("word: [{}]", trimstr(word)); }

  Output:

  $ ./working Comma-separated words: this, that, other word: [this] word: [that] word: [other]

  Because this code runs after the numbers code, and because cin is messy, there may still be a line ending in the buffer. The while(line.empty()) loop will optionally eat any empty lines.

  We use a stringstream object to process the words, so we don't have to do it with cin . This allows us to use getline() to get one line without waiting for the end-of-file state.

  Then, we call getline() on the stringstream object to parse out words separated by commas. This gives us words but with leading and/or trailing whitespace. We use the trimstr() function from the Trim whitespace from strings recipe in this chapter to trim the whitespace.

How it works

The std::cin class is more useful than it may appear, but it can be a challenge to use. It tends to leave line endings on the stream, and in the case of errors, it can end up ignoring input.

The solution is to use getline() and, when necessary, put the line into a stringstream for convenient parsing.

Count words in a file

By default, the basic_istream class reads one word at a time. We can take advantage of this property to use an istream_iterator to count words.

How to do it

This is a simple recipe to count words using an istream_iterator :

• We'll start with a function to count words using an istream_iterator object:

  size_t wordcount(auto& is) {     using it_t = std::istream_iterator<string>;     return std::distance(it_t(is), it_t()); }

  The distance() function takes two iterators and returns the number of steps between them. The using statement creates an alias it_t for the istream_iterator class with a string specialization. We then call distance() with an iterator, initialized with the input stream it_t(is) , and another with the default constructor, which gives us an end-of-stream sentinel.

• We call wordcount() from main() :

  int main() {     const char* fn {"the-raven.txt"};     auto infile = std::ifstream(fn, std::ios_base::in);     println("There are {} words in the file.",       wordcount(infile)); }

  This calls wordcount() with our fstream object and prints the number of words in the file. Using the text of Edgar Allan Poe's The Raven , we get this output:

  There are 1068 words in the file.

How it works

Because basic_istream defaults to word-by-word input, the number of iterations in a file will be the number of words. We use the distance() function to measure the number of steps between the two iterators. The distance between the beginning iterator and the sentinel iterator resolves to the number of words in the file.

Initialize complex structures from file input

One strength of the input stream is its ability to parse different types of data from a text file and convert them to their corresponding fundamental types. Here's a simple technique for importing data into a container of structures using an input stream.

How to do it

In this recipe we'll take a data file and import its disparate fields into a vector of structs. The data file represents cities with their populations and map coordinates.

• This is cities.txt , the data file we'll read:

  Las Vegas 661903 36.1699 -115.1398 New York City 8850000 40.7128 -74.0060 Berlin 3571000 52.5200 13.4050 Mexico City 21900000 19.4326 -99.1332 Sidney 5312000 -33.8688 151.2093

  The city name is on a line by itself. The second line is population, followed by longitude and latitude. This pattern repeats for each of the five cities.

• We'll define our filename in a constant so we can open it later:

  constexpr const char * fn {"cities.txt"};

  Here's a City struct to hold the data:

  struct City {     string name;     unsigned long population;     double latitude;     double longitude; };

• We would like to read the file and populate a vector of City objects:

  vector<City> cities;

  Here's where the input stream makes this easy. We can simply overload the operator>> for istream like this:

  std::istream&operator>>(std::istream& in, City& c) {     in >> std::ws;     std::getline(in, c.name);     in >> c.population >> c.latitude >> c.longitude;     return in; }

  The std::ws input manipulator discards leading whitespace from the input stream.

  We use getline() to read the city name, as it could be one or more words.

  This leverages the >> operator for the population ( unsigned long ), and latitude and longitude (both double ) elements to populate the correct type.

  Now we can open the file and use the >> operator to read the file directly into the vector of City:

  ifstream infile(fn, std::ios_base::in); if(!infile.is_open()) {     println("failed to open file {}", fn);     return 1; } for(City c{}; infile >> c;) cities.emplace_back(c);

• We can display the vector using println() :

  for (const auto& [name, pop, lat, lon] : cities) {     println("{:.<15} pop {:<10} coords {}, {} ",         name, make_commas(pop), lat, lon); }

  Output:

  $ ./initialize_container < cities.txt Las Vegas...... pop 661,903    coords 36.1699, -115.1398 New York City.. pop 8,850,000  coords 40.7128, -74.006 Berlin......... pop 3,571,000  coords 52.52, 13.405 Mexico City.... pop 21,900,000 coords 19.4326, -99.1332 Sidney......... pop 5,312,000  coords -33.8688, 151.2093

  The make_commas() function is from of this book:

  string make_commas(const uint64_t num) {     auto s = std::to_string(num);     for (auto i = s.size(); i > 3; i -= 3) {         s.insert(i - 3, ",");     }     return s; }

How it works

The heart of this recipe is the istream class operator>> overload:

std::istream&operator>>(std::istream& in, City& c) {     in >> std::ws;     std::getline(in, c.name);     in >> c.population >> c.latitude >> c.longitude;     return in; }

By specifying our City class in the function header, this function will be called every time a City object appears on the right-hand side of an input stream >> operator:

City c{}; infile >> c;

This allows us to specify exactly how the input stream reads data into a City object.

There's more…

When you run this code on a Windows system, you'll notice that the first word of the first line gets corrupted. That's because Windows always includes a byte order mark (BOM) at the head of any UTF-8 file. So, when you read a file on Windows, the BOM will be included in the first object you read. The BOM is anachronistic, but as of this writing there is no way to stop Windows from employing it.

The solution is to call a function that checks the first three bytes of a file for the BOM. The BOM for UTF-8 is EF BB BF. Here's a function that searches for, and skips, a UTF-8 BOM:

// skip BOM for UTF-8 on Windows void skip_bom(auto& fs) {     const unsigned char boms[]{ 0xef, 0xbb, 0xbf };     bool have_bom{ true };     for(const auto& c : boms) {         if((unsigned char)fs.get() != c) have_bom = false;     }     if(!have_bom) fs.seekg(0);     return; }

This reads the first three bytes of the file and checks them for the UTF-8 BOM signature. If any of the three bytes do not match, it resets the input stream to the beginning of the file. If the file has no BOM, there's no harm done.

You simply call this function before you begin reading from a file:

int main() {     ...     ifstream infile(fn, std::ios_base::in);     if(!infile.is_open()) {         cout << format("failed to open file {}\n", fn);         return 1;     }     skip_bom(infile);     for(City c{}; infile >> c;) cities.emplace_back(c);     ... }

This will ensure that the BOM is not included in the first string of the file.

Customize a string class with char_traits

The string class is an alias of the basic_string class, with the signature:

class basic_string<char, std::char_traits<char>>;

The first template parameter provides the type of a character. The second template parameter provides a character traits class, which provides basic character and string operations for the specified character type. We normally use the default char_traits class.

We can modify the behavior of a string by providing our own custom character traits class.

How to do it

In this recipe we will create a character traits class for use with basic_string that will ignore case for comparison purposes.

• First, we'll need a function to convert characters to a common case. We'll use lowercase here, but it's an arbitrary choice. Uppercase would work just as well:

  constexpr char char_lower(const char& c) {     if (c >= 'A'&&c<= 'Z') return c + ('a' - 'A');     else return c; }

  This function must be constexpr (for C++20 and later), so the existing std::tolower() function won't work here. Fortunately, it's a simple solution to a simple problem.

• Our traits class is called ci_traits ( ci stands for case-independent). It inherits from std::char_traits<char> :

  class ci_traits : public std::char_traits<char> { public:     ... };

  The inheritance allows us to override only the functions that we need.

• The comparison functions are called lt() , for less-than, and eq() , for equal-to:

  static constexpr bool lt(char_type a, char_type b) noexcept {     return char_lower(a) < char_lower(b); } static constexpr bool eq(char_type a, char_type b) noexcept {     return char_lower(a) == char_lower(b); }

  Notice that we compare using the lowercase versions of characters.

• There's also a compare() function, that compares two C-strings. It returns +1 for greater-than, ‑1 for less-than, and 0 for equal-to. We can use the spaceship <=> operator for this:

  static constexpr int compare(const char_type* s1,         const char_type* s2, size_t count) {     for (size_t i {0}; i < count; ++i) {         auto diff { char_lower(s1[i]) <=> char_lower(s2[i]) };         if(diff > 0) return 1;         if(diff < 0) return -1;         }     return 0; }

• Finally, we need to implement a find() function. This returns a pointer to the first instance of a found character, or nullptr if not found:

  static constexpr const char_type* find(const char_type* p,         size_t count, const char_type& ch) {     const char_type find_c{ char_lower(ch) };     for (size_t i {0}; i < count; ++i) {         if(find_c == char_lower(p[i])) return p + i;     }     return nullptr; }

• Now that we have a ci_traits class, we can define an alias for our string class:

  using ci_string = std::basic_string<char, ci_traits>; In our main() function, we define a string and a ci_string: int main() {     string s {"Foo Bar Baz"};     ci_string ci_s {"Foo Bar Baz"};     ...

• Let's compare two ci_string objects with different cases:

  ci_string compare1{"CoMpArE StRiNg"}; ci_string compare2{"compare string"}; if (compare1 == compare2) {     println("Match! {} == {}", compare1, compare2); } else {     println("no match {} != {}", compare1, compare2); }

  Output:

  Match! CoMpArE StRiNg == compare string

  The comparison works as expected.

• Using the find() function on the ci_s object, we search for a lowercase b and find an uppercase B:

  size_t found = ci_s.find('b'); cout << format("found: pos {} char {}\n", found, ci_s[found]);

  Output:

  found: pos 4 char B

How it works

This recipe works by replacing the std::char_traits class in the template specialization of the string class with a ci_traits class of our own. The basic_string class uses the traits class for its fundamental character-specific functions, like comparisons and searching. When we replace it with our own class, we can change these fundamental behaviors.

There's more…

We can also override the assign() and copy() member functions to create a class that stores lowercase characters:

class lc_traits : public std::char_traits<char> { public:     static constexpr void assign( char_type& r,             const char_type& a ) noexcept {         r = char_lower(a);     }     static constexpr char_type* assign( char_type* p,             std::size_t count, char_type a ) {         for (size_t i{}; i < count; ++i) p[i] = char_lower(a);         return p;     }     static constexpr char_type* copy(char_type* dest,             const char_type* src, size_t count) {         for (size_t i{0}; i < count; ++i) {             dest[i] = char_lower(src[i]);         }         return dest;     } };

Now we can create an lc_string alias, and the object stores lowercase characters:

using lc_string = std::basic_string<char, lc_traits>; ... lc_string lc_s{"Foo Bar Baz"}; println("lc_string: {}", lc_s);

Output:

lc_string: foo bar baz

Parse strings with regular expressions

Regular Expressions (commonly abbreviated as regex ) are often used for lexical analysis and pattern-matching on streams of text. They are common in Unix text processing utilities, such as grep , awk , and sed , and are an integral part of the Perl language. There are a few common variations on the syntax. A POSIX standard was approved in 1992, while other common variations include Perl and ECMAscript (JavaScript) dialects. The C++ regex library defaults to the ECMAscript dialect.

The regex library was first introduced to the STL with C++11. It can be very useful for finding patterns in text files.

To learn more about Regular Expression syntax and usage, I recommend the book, Mastering Regular Expressions , by Jeffrey Friedl.

How to do it

For this recipe we will extract hyperlinks from an HTML file. A hyperlink is coded in HTML like this:

<a href="http://example.com/file.html">Text goes here</a>

We will use a regex object to extract both the link and the text, as two separate strings.

• Our example file is called "the‑end.html" . It's taken from my web site ( ), and included in the GitHub repository:

  const char * fn {"the-end.html"};

• Now we define our regex object with a regular expression string:

  const std::regex     link_re {"<a href=\"([^\"]*)\"[^<]*>([^<]*)</a>"};

  Regular expressions can look intimidating at first, but actually they're rather simple.

  This is parsed as follows:

  • Match the whole string
  • Find the substring <a href= "
  • Store everything up to the next " as sub-match 1
  • Skip past the > character
  • Store everything up to the string </a> as sub-match 2
• Now we read our file entirely into a string:

  string in{}; std::ifstream infile(fn, std::ios_base::in); for (string line {}; getline(infile, line);) in += line;

  This opens the HTML file, reads it line-by-line, and appends each line to the string object in .

• To extract the link strings, we set up a sregex_token_iterator to step through the file and extract each of the matched elements:

  std::sregex_token_iterator it {in.begin(), in.end(),     link_re, {1, 2}};

  The 1 and 2 correspond to the sub-matches in the regular expression.

• We have a corresponding function to step through the results with the iterator:

  template<typename It> void get_links(It it) {     for (It end_it {}; it != end_it; ) {         const string link {*it++};         if (it == end_it) break;         const string desc {*it++};         cout << format("{:.<24} {}\n", desc, link);     } }

  We call the function with the regex iterator:

  get_links(it);

  And we get this result with our descriptions and links:

  Bill Weinman............ https://bw.org/ courses................. https://bw.org/courses/ music................... https://bw.org/music/ books................... https://packt.com/ back to the internet.... https://duckduckgo.com/

How it works

The STL regex engine operates as a generator that evaluates and yields one result at a time. We set up the iterator using sregex_iterator or sregex_token_iterator . sregex_token_iterator supports sub-matches. sregex_iterator does not.

The parentheses in our regex serve as sub-matches , numbered 1 and 2 respectively.

const std::regex link_re {"<a href=\"([^\"]*)\"[^<]*>([^<]*)</a>"}

Figure 7.1: Regular expression with sub-matches

This allows us to match a string and use parts of that string as our results:

sregex_token_iterator it {in.begin(), in.end(), link_re, {1, 2}};

The sub-matches are numbered beginning with 1 . Sub-match 0 is a special value that represents the entire match.

Once we have our iterator, we use it as we would any iterator:

for (It end_it {}; it != end_it; ) {     const string link {*it++};     if (it == end_it) break;     const string desc {*it++};      println("{:.<24} {} ", desc, link); }

This simply steps through our results via the regex iterator giving us the formatted output:

Bill Weinman............ https://bw.org/ courses................. https://bw.org/courses/ music................... https://bw.org/music/ books................... https://packt.com/ back to the internet.... https://duckduckgo.com/

Subscribe to Deep Engineering

Join thousands of developers and architects who want to understand how software is changing, deepen their expertise, and build systems that last.

Deep Engineering is a weekly expert-led newsletter for experienced practitioners, featuring original analysis, technical interviews, and curated insights on architecture, system design, and modern programming practice.

Scan the QR or visit the link to subscribe for free.