Measuring the size of sets

People measure sets all the time. A bag of marbles is measured by counting them, a road by its length, a field by its area, a metal bar by its weight. These are four different ways of attaching a number to a collection of things, and they look nothing alike, yet they obey the same few rules. This lesson finds those rules, turns them into the definition of a measure, and explains why one of them, countable additivity, has to be stronger than it first looks. Everything else in this track is built on that definition. The lessons that follow ask which sets can be measured at all, how length itself is constructed, and how to add up values of a function the way we add up sizes of sets.

Four ways to measure a set

A set is a collection of things, called its elements. To measure a set is to attach a number to it that says how big it is. Four examples will be used throughout this track, so they are worth setting up carefully. In each of them there is an underlying set $\Omega$ that contains everything we may talk about, and a rule $\mu$ that takes a part $A$ of $\Omega$ and returns the number $\mu(A)$, read "mu of $A$".

Counting on the faces of a die. Let $\Omega = \{1, 2, 3, 4, 5, 6\}$ and let $\mu(A)$ be the number of faces in $A$. The even faces $\{2, 4, 6\}$ have $\mu = 3$, and $\mu(\Omega) = 6$.

Length on the interval $[0, 1]$. Let $\Omega = [0, 1]$ and let $\mu(A)$ be the length of $A$. The interval $[0.1, 0.5]$ has length $0.5 - 0.1 = 0.4$. This measure is written $\lambda$ from now on.

Area in the unit square. Let $\Omega = [0, 1] \times [0, 1]$ and let $\mu(A)$ be the area of $A$. The whole square has area 1. A disc of radius $0.1$ inside it has area $\pi \cdot 0.1^2 \approx 0.0314$, and the rectangle $[0.2, 0.6] \times [0.1, 0.4]$ has area $0.4 \cdot 0.3 = 0.12$.

Mass on a rod. Picture a thin rod of length 1 that gets thicker towards its right end. At position $x$ the rod has density $\rho(x) = 2x$, meaning mass per unit of length. A short piece of length $\Delta$ near position $x$ then weighs about $2x\,\Delta$, and adding up the pieces of a stretch $[a, b]$ is the familiar integral

The whole rod has mass $1^2 - 0^2 = 1$. The left half $[0, 0.5]$ has mass $0.25$ and the right half $[0.5, 1]$ has mass $0.75$: the two halves have the same length, but the right half is three times as heavy.

In all four examples the number belongs to a set of positions and not to one position. The weight of a single point of the rod can be asked for, but the answer is 0; the interesting questions are about stretches, regions and groups of faces.

Rules every size obeys

What do counting, length, area and mass have in common? Only a short list of rules, and three of them are all that is needed.

Sizes are not negative: $\mu(A) \ge 0$. Without this rule a bigger set could be smaller than a part of itself.

The empty set has size 0: $\mu(\emptyset) = 0$. A set with no elements has nothing to count, no length, no area and no mass. This rule also follows from the next one whenever some set has a finite size, because a set $A$ together with the empty set is still $A$, and the two are disjoint, which means they have no element in common. The next rule then gives $\mu(A) = \mu(A) + \mu(\emptyset)$, which forces $\mu(\emptyset) = 0$. The rule is stated separately to rule out a silly rival that gives every set, even the empty one, the size $\infty$.

Disjoint pieces add up: if $A$ and $B$ are disjoint, then

This is the rule that makes a measure useful. On the die, the pieces $\{1, 2\}$ and $\{5, 6\}$ have sizes 2 and 2, and their union $\{1, 2, 5, 6\}$ has size 4. For length, $[0, 0.2]$ and $[0.5, 0.6]$ have lengths $0.2$ and $0.1$, and together $0.3$. On the rod, the halves $[0, 0.5]$ and $(0.5, 1]$ weigh $0.25$ and $0.75$, and together $1$. Because of additivity, the size of a complicated set follows from the sizes of simple pieces that it can be cut into; without it, a size would be an arbitrary label on each set, and knowing the sizes of the parts would say nothing about the whole. By repeating the rule, it also holds for any finite number of pairwise disjoint sets.

The word "disjoint" matters. The sets $[0, 0.5]$ and $[0.3, 0.8]$ have lengths $0.5$ and $0.5$, but their union $[0, 0.8]$ has length $0.8$, not $1$, because the stretch $[0.3, 0.5]$ belongs to both and may be counted only once. The diameter of a set, the distance between its two farthest points, gives $0$ to each of the single points $\{0.1\}$ and $\{0.9\}$ but $0.8$ to the pair $\{0.1, 0.9\}$; it is not a measure in the sense of this lesson.

Consequences of additivity

The three rules have many consequences, and the first few come from one trick: cut a set into disjoint pieces and add. Below, all sets are taken from the collection of sets that is being measured, and it is assumed that this collection contains the unions, intersections and differences that appear. The next lesson, "σ-algebras and Borel sets", says exactly what the collection has to look like.

Monotonicity. Suppose $A \subseteq B$, so that $A$ is a part of $B$. Then $B$ is the union of $A$ and the rest, $B \setminus A$, the elements of $B$ that are not in $A$, and these two are disjoint:

A part is never larger than the whole. If $\mu(A)$ is finite, the same equation can be solved for the size of the rest:

The condition on $\mu(A)$ is needed. The half-line $A = [1, \infty)$ is part of $B = [0, \infty)$, both have infinite length, and the rest $B \setminus A = [0, 1)$ has length 1. The formula would read $\infty - \infty$, which can be any number you like.

Inclusion–exclusion. Take two overlapping intervals, $A = [0.1, 0.5]$ and $B = [0.3, 0.8]$, with lengths $\mu(A) = 0.4$ and $\mu(B) = 0.5$. They overlap in $A \cap B = [0.3, 0.5]$, of length $0.2$. Cut the union into three disjoint pieces: the part only in $A$, the overlap, and the part only in $B$. Then

Adding these two equations counts the overlap twice and the other two pieces once each. The union is exactly those three pieces, so

In words: adding the sizes of $A$ and $B$ counts the overlap twice, so it has to be subtracted once. With the numbers above, $0.4 + 0.5 - 0.2 = 0.7$, which is the length of $[0.1, 0.8]$. The formula is valid when the sizes are finite. On the die, $A = \{2, 4, 6\}$ and $B = \{4, 5, 6\}$ have size 3 each and share the two faces $\{4, 6\}$, so their union $\{2, 4, 5, 6\}$ has size $3 + 3 - 2 = 4$.

Subadditivity. The overlap has a non-negative size, so inclusion–exclusion gives $\mu(A \cup B) \le \mu(A) + \mu(B)$. For the intervals above, $0.7 \le 0.4 + 0.5 = 0.9$. The size of a union is at most the sum of the sizes, with equality exactly when the overlap has size 0. This is often called the union bound, and repeating it extends it to any finite number of sets.

Countable additivity

Additivity as stated deals with two sets at a time, and therefore with any finite number of sets. Limits, however, involve infinitely many steps, and it turns out that the rule has to reach that far. Here is an example where infinitely many pieces appear naturally. Cut $[0, 1)$ at $\tfrac12$, then cut what is left at $\tfrac34$, then at $\tfrac78$, and so on, always keeping the left piece. The $k$-th piece is

The first three pieces are $[0, \tfrac12)$, $[\tfrac12, \tfrac34)$ and $[\tfrac34, \tfrac78)$, with lengths $\tfrac12$, $\tfrac14$ and $\tfrac18$. After $n$ pieces, finite additivity says that a total length of $1 - 2^{-n}$ has been used; for $n = 10$ that is $0.9990$. The pieces never run out, and together they fill all of $[0, 1)$, whose length is 1. Finite additivity says nothing about "all pieces together", yet the natural thing to ask is that the infinite sum $\tfrac12 + \tfrac14 + \tfrac18 + \cdots = 1$ gives the length of the whole.

A measure is therefore required to be countably additive: for every sequence of pairwise disjoint sets $A_1, A_2, A_3, \ldots$, that is, a list in which there is a first set, a second set and so on,

Is the extra strength necessary, or is finite additivity enough? An example shows that it is not enough. For a set $A$ of positive integers, count how many of its elements are among the first $n$ integers and divide by $n$. If this fraction settles down to a limit as $n$ grows, call the limit the natural density $d(A)$. The even numbers have density $\tfrac12$: among the first 1,000 integers exactly 500 are even. The multiples of 3 have density $\tfrac13$. A single integer such as $\{7\}$ has density 0, since $1/n$ tends to 0, while the set $\mathbb{N}$ of all positive integers has density 1. For sets that have a density, disjoint pieces do add: the even and the odd numbers have densities $\tfrac12$ and $\tfrac12$, and together they have density 1.

So natural density follows the rules of the previous section, except that not every set has a density. It fails countable additivity badly. The set $\mathbb{N}$ is the union of the countably many single-number sets $\{1\}, \{2\}, \{3\}, \ldots$, which are disjoint and each have density 0, but $d(\mathbb{N}) = 1 \ne 0 + 0 + 0 + \cdots$. A size like that cannot be trusted with limits: cutting a set into more and more pieces and adding their sizes may give a different answer from the size of the whole. This is why measure theory insists on countable additivity.

A measure takes a whole set as its input. The length of $[0, 1]$ is not obtained by adding something attached to each point; in counting and in point masses it is, but in length, area and density it is not, since every single point has length 0.

The rule stops at countable families, and has to. The interval $[0, 1]$ is the union of its points, and each point has length 0, but the length of $[0, 1]$ is 1. A family of pieces that cannot be listed one after another, called uncountable, can add up to something that the sizes of the pieces do not predict. Countable additivity is the strongest form of the rule that length, area and mass satisfy.

The definition of a measure

Everything so far fits into one definition. A measure on a set $\Omega$ is a function $\mu$, defined on a collection of subsets of $\Omega$, whose values lie in $[0, \infty]$, such that

• $\mu(\emptyset) = 0$, and
• $\mu$ is countably additive: $\mu\big(\bigcup_k A_k\big) = \sum_k \mu(A_k)$ for every sequence of pairwise disjoint sets $A_1, A_2, \ldots$ of the collection.

Non-negativity is built into the range $[0, \infty]$, and finite additivity is a special case of the second rule: for disjoint $A$ and $B$, take the sequence $A, B, \emptyset, \emptyset, \ldots$ and use the first rule on the empty sets,

The value $\infty$ is allowed because it is needed. The counting measure of the positive integers is infinite, and so is the length of the whole real line. Arithmetic with it follows the obvious rule that $\infty$ plus anything is $\infty$, and a series of non-negative terms that does not converge has the sum $\infty$.

The phrase "a collection of subsets" hides a real question: which sets get a size? When $\Omega$ is finite, the answer is simple, every subset. On $[0, 1]$ it is not, and for the moment it is enough to know that the collection contains the intervals and everything that can be built from them by taking complements, differences and countable unions. The lesson "σ-algebras and Borel sets" turns this into a precise definition and shows why we cannot always allow every subset.

Examples of measures

The measures of the first section fit the definition, and there are others.

• Counting measure. $\mu(A)$ is the number of elements of $A$, and $\infty$ if there are infinitely many. Countable additivity is clear, since the elements of a union of disjoint sets are counted once each.
• Dirac measure. Fix a point $x$ of $\Omega$. The measure $\delta_x$ puts all its mass at that one point: $\delta_x(A) = 1$ if $x \in A$ and $\delta_x(A) = 0$ otherwise. On $[0, 1]$, $\delta_{0.5}([0.4, 0.6]) = 1$ and $\delta_{0.5}([0.6, 0.9]) = 0$. On the die, $\delta_3(\{1, 2, 3\}) = 1$ and $\delta_3(\{2, 4, 6\}) = 0$. Disjoint sets cannot both contain $x$, so additivity holds.
• Weighted point masses. Put weights $w_1, w_2, \ldots$ on points $x_1, x_2, \ldots$ and set $\mu = \sum_i w_i \delta_{x_i}$, which means that $\mu(A)$ is the sum of the weights of the points that lie in $A$. Counting measure is the case where all weights equal 1. A loaded die, with weight $0.5$ on the face 6 and $0.1$ on each other face, has $\mu(\{2, 4, 6\}) = 0.1 + 0.1 + 0.5 = 0.7$ and $\mu(\Omega) = 1$.
• Length and area. The measures $\lambda$ on $[0, 1]$ and area on the unit square. Showing that they are countably additive on all the sets one would like to measure is not trivial, and a later lesson of this track does it; until then only intervals, rectangles and discs are needed.
• A measure with a density. Given a function $\rho \ge 0$ on $[0, 1]$, set $\mu(A) = \int_A \rho(x)\,dx$. The rod of the first section has $\rho(x) = 2x$, and length is the case $\rho = 1$. Unlike a point mass, a measure with a density gives every single point the mass 0: the mass is spread out thinly instead of being concentrated.

The figure puts one pair of sets through four of these measures. It measures two intervals $A = [a_1, a_2]$ and $B = [b_1, b_2]$ of $[0, 1]$ under length, under counting on the ten points $0.05, 0.15, \ldots, 0.95$, under weighted points with mass $0.5$ at $0.2$, $0.3$ at $0.45$ and $0.2$ at $0.9$, and under the rod density $2x$. Choose the measure with the buttons, move the four sliders to change $A$ and $B$, and compare the readouts. With the default $A = [0.1, 0.5]$ and $B = [0.3, 0.8]$, length gives $0.4$, $0.5$, $0.2$ and $0.7$, as computed above. Counting gives $4$, $5$, $2$ and $7$: $A$ contains the points $0.15, 0.25, 0.35, 0.45$, and $B$ contains $0.35, 0.45, \ldots, 0.75$. Weighted points give $0.8$, $0.3$, $0.3$ and $0.8$, because $A$ contains the masses at $0.2$ and $0.45$ and $B$ only the one at $0.45$. The rod gives $0.24$, $0.55$, $0.16$ and $0.63$. In every case the last readout equals $\mu(A \cup B)$.

Two things are worth trying. Move $B$ to the right until it no longer touches $A$: the overlap has size 0 and the sizes simply add. Then set both ends of $A$ to $0.2$, so that $A$ is a single point: length, counting and the rod all give 0, while the weighted points still give $0.5$. Some measures see single points and some do not.

Finite, infinite and probability measures

The size of the whole space, $\mu(\Omega)$, is the total mass of the measure. A measure is finite if its total mass is finite and infinite otherwise. Counting on the die has total mass 6, length on $[0, 1]$ has total mass 1, and the rod has total mass 1. Counting on $\mathbb{N}$ and length on the real line have total mass $\infty$.

A finite measure with positive total mass can always be rescaled so that its total mass is exactly 1: divide every size by the total. Counting on the die divided by 6 gives each face the size $\tfrac16$, and the even faces $\tfrac36 = \tfrac12$. The rescaled version is again a measure, because dividing by a constant keeps every rule, and it has total mass 1. A measure of total mass 1 is called a probability measure. The loaded die of the previous section, the rod and length on $[0, 1]$ are already probability measures. In the probability part of this track, $\mu(A)$ is read as the chance that something lands in $A$, and the whole subject is measure theory with this one extra normalisation. For now, a probability measure is just a measure whose total mass happens to be 1.

Finite measures allow subtraction, $\mu(B \setminus A) = \mu(B) - \mu(A)$ for any $A \subseteq B$, since every size is finite; with infinite measures, subtraction and limits need care, as the next section shows.

Continuity from below and above

Countable additivity has a consequence for sequences of growing and shrinking sets. Start with sets that grow, $A_1 \subseteq A_2 \subseteq A_3 \subseteq \cdots$, and let $A$ be their union. To use countable additivity, cut $A$ into disjoint rings: $D_1 = A_1$, and $D_k = A_k \setminus A_{k-1}$ for $k \ge 2$, the part that the $k$-th set adds. The rings are disjoint, their union is $A$, and the first $n$ of them make up $A_n$. Therefore

This is continuity from below: the sizes of growing sets grow to the size of the limit set. For example, $A_n = [1/n, 1]$ grows as $n$ grows, and its union is $(0, 1]$. The lengths are $1 - 1/n$, which are $0.9$ for $n = 10$ and $0.99$ for $n = 100$, and they tend to $1 = \lambda((0, 1])$. The missing point 0 has length 0, so it makes no difference.

Shrinking sets, $B_1 \supseteq B_2 \supseteq \cdots$ with intersection $B$, behave the same way, provided the first set has finite size. Apply the previous argument to the growing sets $B_1 \setminus B_n$, whose union is $B_1 \setminus B$, and subtract from the finite number $\mu(B_1)$. This gives continuity from above: $\mu(B_n) \to \mu(B)$ when $\mu(B_1) < \infty$. For example, $B_n = [0, 1/n]$ has length $1/n$, which tends to 0, the length of the intersection $\{0\}$.

The condition of finiteness cannot be dropped. The sets $B_n = [n, \infty)$ shrink and every one of them has length $\infty$, but nothing lies in all of them: the intersection is empty and has length 0, while the limit of the lengths is $\infty$.

The same trick of cutting into disjoint rings also gives the union bound for countably many sets that overlap. Let $A_1, A_2, \ldots$ be arbitrary, and cut the union into the disjoint parts $D_k = A_k \setminus (A_1 \cup \cdots \cup A_{k-1})$. Each $D_k$ is part of $A_k$, so $\mu(D_k) \le \mu(A_k)$ by monotonicity, and countable additivity gives $\mu\big(\bigcup_k A_k\big) = \sum_k \mu(D_k) \le \sum_k \mu(A_k)$.

What this lesson leaves open

A measure is a number attached to sets, obeying three rules that every size obeys, with additivity strengthened to countable families. The definition leaves three questions, and the following lessons of this track answer them one by one. Which collections of sets can carry a measure? The next lesson, "σ-algebras and Borel sets", answers this. Does a countably additive length exist on all those sets? And can a measure be used to add up not just sizes of sets, but values of a function? Each of these needs the definition above, which is why it comes first.

Summary

• Counting, length, area and mass are measures: rules $\mu$ that give a set $A$ a number $\mu(A) \in [0, \infty]$.
• The rules are $\mu(\emptyset) = 0$ and additivity for disjoint sets; they give monotonicity, $\mu(B \setminus A) = \mu(B) - \mu(A)$ for finite $\mu(A)$, inclusion–exclusion and the union bound.
• Additivity is demanded for countably many disjoint sets, because natural density is finitely additive but fails for $\mathbb{N} = \{1\} \cup \{2\} \cup \cdots$; it cannot be demanded for uncountably many, since $[0, 1]$ is the union of its points of length 0.
• Examples: counting measure, the Dirac measure $\delta_x$, weighted points $\sum w_i \delta_{x_i}$, length, area and a measure with a density $\rho$; a measure of total mass 1 is a probability measure.
• Countable additivity gives continuity from below, and from above when the first set has finite size; the sets $[n, \infty)$ show why finiteness is needed.