Chapter 1
Performance Measures
Reading: The corresponding chapter in the 2nd edition is Chapter 2, in the 3rd edition it is Chapter 4 and in the 4th edition it is Chapter 1.

When selecting a computer, there are different criteria a user might want to consider
• Whether the application one wants to run is available for that computer
• Energy consumption
• Cost
• Often, the most important factor is: performance (good resolution, smooth execution, not too slow)
The goal of this chapter is to define more precisely what we mean by “performance” and how we go about measuring it.

1.1

Measuring performance

What do we mean if we say that one computer has a performance that is better than another computer? The answer is not obvious. That defining performance is not unambiguous is illustrated by the following example.
Example 1.1.1
The following table lists the characteristics of three different airplanes.

Boeing 747
Concorde
Douglas DC-8

# of passengers
470
146
132
1

Range
4150
4000
8720

Speed
610
1360
544

2

CHAPTER 1. PERFORMANCE MEASURES

(a) If you are one of the passengers, who wants to get to her/his destination as fast as possible, your best pick would be the airplane with the highest speed, i.e., the
Concorde.
(b) If you are the president of the airline company, and you want to maximize the number of passengers that you can transport in a day between Paris and New York (3, 000 miles route), your best pick would be the Boeing 747: although it is about two times slower than the Concorde, so the Concorde could make two round trips in the time the Boeing makes a single roundtrip, the Boeing can transport more than three times as many people, per flight, so it could still transport the most passengers per day.
(c) If you want to buy a private airplane and your primary concern is to be able to fly direct from Paris to San Diego (5, 500 miles), without stopping, than the Douglas
DC-8 would be your best pick. The range of the other two airplanes is not sufficient to fly directly from Paris to San Diego.
As the example illustrates, the optimal choice can vary depending on what criterion one uses to define “optimal”. Similarly, there are different ways to define “performance” for computers. For example, one could define it as:
• The “response time”, also called “execution time” or “latency”: the time it takes from start to completion of a task (in the airplane analogy: fly one passenger from Paris to
New York; and, thus, choose the Concorde)
• The “throughput”: the total amount of work completed in a given time interval (in the airplane analogy: the total number of passengers that can be transported in a given time interval, for example, one day; and, this, choose the Boeing 747)
In this class, we will focus on execution time to measure performance.

1.2
1.2.1

Execution time
Definition

Execution time is the time between starting and completing a task. We will denote execution time as ET .

1.2.2

Performance

Performance is denoted by P F and is related to the execution time ET as
1
.
ET
Indeed: to maximize the performance, we would want to minimize the execution time, for a given task.
PF =

1.3. MEASURING EXECUTION TIME

3

Example 1.2.1
If the performance of machine A is ten times better than machine B, what is the relation between their execution times?
1
1
Answer: Since P FA = 10 · P FB , we have ETA = 10 · ETB and thus ETB = 10 · ETA . So, A being ten times better than B indeed implies that B needs ten times more time than A to execute a given task, or B is ten times slower than A.

1.2.3

CPU Time

Measuring the time that elapses from the start to the completion of a task would include the time needed to execute the task on the CPU, the time to access the hard drive when needed, the time waiting for I/O (input/output) from the user, which could be extremely long, the time required for OS (operation system) overhead (often,