Using the timeit Module

One of the easiest ways to benchmark Python code is with Python’s timeit module. In the words of Python’s documentation: “This module provides a simple way to time small bits of Python code.”

Usually, the timeit module is used from the command line to measure small snippets of code, but since we’re going to use it to measure more complex algorithms, we’re going to call the module from within an actual code file.

Let’s start with a basic example to see how timeit works. In Python, you can create an array containing one million integers (0 through 999999) with the following code:

(Note that Python allows us to use underscores to make long numbers more readable.)

Let’s use timeit to see how quickly this code runs. Here’s the code that does this:

I’ll explain what the code means shortly, but first, let’s run it.

I’ve saved this code inside a file called bench_first_example.py. If I now run python bench_first_example.py from the command line, I get the following output:

This is the number of seconds that it took my computer to run my array-generating code. Wow, it ran in a fraction of a second! Specifically, it ran in 0.121737003326 seconds, which is roughly one-eighth of a second.

When I change my code to generate an array that contains only 100 integers, timeit spits out this result:

If you don’t look too carefully, this may seem to be saying that my code took about 1.5 seconds to run. But this would make no sense. Why would creating an array of 100 elements be much slower than creating an array containing 1,000,000 elements?

However, notice the e-05 at the end of this output. This is scientific notation, and is a shorthand way of expressing this number:

This is about 10,000 times faster than the 0.121737003326 seconds it took to generate 1,000,000 integers.

As a quick tip, if you want to convert scientific notation into “regular” notation, there’s an easy way to do so. Here’s an example:

The .08 represents how many digits you want to see, which in this case is 8 digits. The previous code outputs this result:

If you want to see, say, 10 digits, you’d run this:

This outputs a result:

Breaking Down the Code

Let’s break down how the timeit-based code works.

First, we import the timeit module:

The next part may seem a little wonky, but timeit demands that we pass in all of the code we’re benchmarking as a string. Here, we store the string in a variable called test_code:

For readability, I used the triple-quote Python syntax for creating a multiline string. I also named the variable test_code, but it could be named anything you want.

We then pass our test_code into the timeit function and print the results:

The timeit function can accept numerous options as parameters. The main argument is stmt, and that’s where we pass in our test_code.

The number argument represents how many times we want to run the test_code. For this example, we’ll run our test_code one time. Soon, I’ll explain why you may want to run your code multiple times, but for now, we’re going to keep number as 1. However, I will point out now that if you don’t pass in any number argument, the default is 1,000,000! So, I always recommend passing in the number function. Otherwise, you may be waiting a long time.

Using timeit to Compare Two Algorithms

Measuring the speed of a single algorithm alone has limited value. Knowing that one algorithm takes, say, two seconds is somewhat meaningless since the same algorithm may run significantly faster or slower on other computers. So, no algorithm can ever be labeled a “two-second” algorithm. This is one of the reasons we’ve always counted steps as a more reliable way to express an algorithm’s speed. After all, the number of steps that an algorithm takes remains consistent no matter which computer it’s run on.

That being said, benchmarking shines when you are trying to compare two or more competing algorithms against each other. This is indeed a useful measurement since as long as you do all of the benchmarking on the same computer, you can find out which algorithm is the fastest in actual time.

Let’s try this out for testing two of Python’s built-in functions. In the previous example, we benchmarked using the append method to create an array of one million ascending integers.

Python has another method for adding values to an array, namely, the insert method. If we want to generate an array of one million integers with this method, we’d run the following code:

With benchmarking, we can determine whether append or insert is a faster method for populating an array.

Here’s my benchmarking code. It’s virtually the same as our code for benchmarking append, except that I’m using the insert approach:

When I benchmark the insert code, I get a result of:

This is almost three times slower than our append benchmarking result, which was 0.121737003326. Based on this benchmarking experiment, it would seem that append is notably faster than insert.

It can be super fun to conduct benchmarking experiments. We assume the role of scientists and measure real, tangible results. It’s so … scientific!