4 Design document

It is worth noting here that there is no set-in-stone order of actions at the early stage of creating a machine learning (ML) system. You can start preparing a design document as soon as you’ve identified the problem and goals (especially if you work in a startup, where the speed of delivery is often more important than following processes). But since this book is presented as a checklist, the list of actions is also displayed in a traditional sequence.

As one of the authors’ managers once said, no fancy recommendation algorithm can beat a customer with a shopping list. These people have a goal and a plan for achieving it. Nothing can stop them.

If you think about it, writing code is just providing a specific set of instructions to achieve a particular goal. In a sense, a design document is a meta-algorithm set to accomplish a specific goal, with the involvement of many subalgorithms. Still, the design document is being regularly challenged by many as either one of the horsemen of bureaucratization or a rudiment used out of inertia.

In this chapter, we will examine the most common myths around the design document. We will introduce and define the concept of antigoals as additional guidelines that lead you toward the project’s objectives, and we will start the practical part of this book, represented by two design documents based on close-to-real-life scenarios.

When the book was in early access, there was a common comment shared by our early readers: “Well, this is good, but that is not how it works in startups.” While agreeing that startups’ delivery cadence is different, we still stick to the idea that the design phase is necessary. It is true that cofounders and early engineers can find their consensus during a coffee break, whereas a massive corporation would waste 6 months on the same scope. We also agree that writing formal docs may be inefficient, but that is not what we advocate for. A simple note with a short description can be enough as soon as you are sure it gets all collaborators on the same page. Ignoring good practices of software and ML engineering is fine while you’re hunting for a prize at a hackathon, but the hackathon style barely works at longer distances.

Typically, such an argument can come either from a person with a lack of flexibility or from an ardent opponent of the design document who is eager to use any argument in their favor.

Practice shows that even for smaller projects, a well-structured design document ensures early identification of potential risks, serves as a reference for future enhancements if the project eventually expands, and, most importantly, helps prevent scope creep when every other stakeholder is tempted to add just one more feature.

Even simple initiatives can benefit from a design document with a proportional level of detail.

At first glance, the process of creating a design document may seem straightforward and simple. In reality, right from the start, you will encounter a whole load of factors that will interfere with the process and set you several steps back if ignored.

Remember: your task is not to create a draft document and convince everyone of its purity and correctness. Your task is to find as many weak points as possible (including motivating your stakeholders to find them) so that eventually, after a number of iterations, you have a document that allows you to start developing your ML system.

The same principle applies to designing ML systems. An ML system is a highly complex machine of interconnected domains that requires thorough preparation when your design document undergoes multiple iterations before implementation. Still, more often than not, a good design document leads to no ML project at all.

Each part of a design document can be viewed as an answer to multiple questions: what are the goals of a potential system, what are the key success criteria, what tech aspects should we focus on, how do we solve a given subproblem, etc. A rookie mistake would be to miss tradeoffs and enumerate endless goals for the system: for example, it should do X, Y, and Z; have high performance; be precise; be easy to maintain and cheap to develop; and be intuitively understandable. Obviously, it’s impossible to successfully fit all the good properties into one system, and you will require an approach to counterbalance this possible excessiveness.

Setting antigoals allows us to strike out the aspects we don’t really care about that much and additionally highlight those we see as crucial. Let’s say we’re building a system that will be used internally, and the output artifacts are various reports to be read by the executive team and analysts. We can assume right off the bat that processing time won’t be critical for such a system—it’s only necessary to make it work fast enough that reports are ready by morning. Thus, “processing time” will be the first to join the list of antigoals so that we don’t bother ourselves with this parameter. Or imagine building a recommendation engine for a boutique store: you sure won’t need to support millions of items if the current number of goods contains only three digits (see figure 4.1), meaning excessive productivity is a no-go for the end solution.

The following example suggests what the lists of goals and antigoals would look like for a boutique store’s recommendation engine:

Antigoals should not be considered the main source of information in your design document but can become a spice that adds a missing flavor, growing into an essential part of the document’s structure.

In what follows, we will introduce you to two fictional cases, each with its own specifics, features, problems, and context. These two cases will form the basis of two different design documents, which will gradually grow and evolve from chapter to chapter, adding more depth and complexity. Eventually we will have two fully formed documents at our disposal.

In this section, we start to outline a design document for a project as it might have been written in real life. For this purpose, we introduce a fictional company, Supermegaretail, a retail company with a demand forecast project to launch.

In section 4.4, we give a very brief example of what the first chapter of a design document can look like. We will include only major topics; otherwise, it would not fit into a single book.

The next section of this chapter is no less important: it gives a practical example of how to review a design document. If you’re new to ML system design, you probably haven’t reached the stage of your career where you have enough experience and credibility to be involved in this kind of working routine. However, stepping up to review your first design doc is just a matter of time, so it’s better to be prepared beforehand, and you will see some practical advice on the reviewing basics.

This fact is essential and quite easily explained. Complex systems require input from many people with diverse expertise and backgrounds. As a design document author, part of your job is to make it more manageable for all the involved parties to navigate. Outlining your doc with chapters and subchapters will help domain experts see where to go from the beginning. Otherwise, the natural reaction for most people when they see a 10+ page doc is to close it and forget it.

Here come the first two critical points: the design doc must be accessible and visible to as many people as possible and easy to navigate for all participants.

As soon as people start reviewing any kind of content, they begin to criticize and offer alternatives. As an author, you want to encourage this type of behavior. After all, what are the chances you had the best and most appropriate design after the first iteration?

Try to derive an explanation for each proposition/fixture, as they could emerge from different conditions:

A popular failure mode for design documents is writing too generically. That is a huge drawback for a design doc, and often it is caused by the fact that a single person may not have enough context to fill in all the gaps. As an initial author, you need to facilitate the others’ inputs—for example, highlight some problematic areas with a lack of required information and encourage the reviewers to add missing parts of the puzzle.

We discussed how to create a design doc and what to expect from reviewers, but because the title of this section is “Reviewing a design document,” let’s try to reverse our suggestions and apply them from the reviewer’s standpoint:

The business is effectively a marketplace: photographers join the platform and upload their shots; customers who are looking for specific images for illustrative purposes (editors, designers, ad professionals) purchase rights for these photos. The marketplace makes money through commission from sales. The company is highly interested in making an effective search system on its website.

We provide part of a raw and poorly written design document based on what we discussed in the previous chapters and comment on it as if we were reviewing the document. This time, text highlighted in italics represents reviewers’ comments.

So why should there be no fear or hesitation in editing or criticizing a design doc at any stage? The answer is that a design doc is truly a living thing.

Usually, the evolution of design docs looks like this:

As soon as your system is implemented, life will expose its flaws, which you will have to address; or product managers decide they need new features, and the system has to be extended; or the government issues a new piece of legislation, which you have to consider; or there is an infrastructure migration or a new use case. You name it. To perform these changes, engineers need to understand the system and read design documents. By that time, a new pattern or technology could arise that perfectly fits the system.

If this is not the case, new features and refactoring need to be reflected in the design doc, bringing us to the design doc evolution mentioned earlier.

That is why a design document is never over. It is a living thing, as long as you have a service it describes. Even if you leave the company, others need to take the banner from you, if they don’t want to end up with a completely unsupportable system.

Rewriting a solid share of the design doc may seem discouraging, but it is something you can benefit from in the long run. For complex systems, it even makes sense to practice the “design it twice” approach—admit that your first design is likely not the best one, and design it twice, taking two radically different approaches. As practice shows, this approach can reveal hidden problems and opportunities. Let us quote A Philosophy of Software Design (Yaknyam, 2018) by John Outerhout (a nice book we recommend to feel the spirit of a good design):

People who prefer building over thinking may feel irritated by this point: “Come on, first you suggest writing docs instead of writing code, and now you suggest doing it over and over again?” Well, it makes little sense to run many iterations once you no longer receive new information, and sometimes you can’t improve the design before some proof of concept is written. However, designing things twice is often a fair tradeoff between agility and preparedness.