Measures and Integrals: Measures! and What Do You Know About Measures?

Written on November 1, 2017

NOTE: This post is automatically translated from Farsi using GPT 5.5

Preface: Before starting the second part of this series of posts, I need to admit that practical examples of measures are not like practical examples of algorithms. Measures are a level of mathematical abstraction that has made it possible to study integration, and especially probability theory, precisely and rigorously. For this reason, their applications are also of the kind that clarify and broaden one’s perspective. What I mean is that by using this abstraction, we can look at some problems from another angle, one that makes solving them easier — or even possible.

Main point: We saw that after changing the perspective on integration from partitioning the domain to partitioning the range of the function, the “measure” trick can, in addition to making integrals over unbounded parts of the domain well-defined, also make it possible to compute new integrals. Well then, we should take “measures” more seriously.

Measures are functions that take a set as input and give a positive real number as output. This means that the domain of measures is a set of sets. For this reason, it would be good to get to know the domain of these functions better.

Sigma-algebras

For the sake of variety, I’ll start this section with dry mathematical definitions.

Definition Suppose $D$ is a set and $2^{D}$ is its power set (the set of all subsets of $D$ ). Any subset $Σ \subseteq 2^{D}$ that satisfies the following three conditions is a sigma-algebra:

$D$ belongs to $Σ$ : $D \in Σ$
$Σ$ is closed under “complementation”: $A \in Σ \Rightarrow D - A \in Σ$
$Σ$ is closed under “countable union”: $A_{1}, A_{2}, A_{3}, \dots \in Σ \Rightarrow \cup_{i = 1}^{\infty} A_{i} \in Σ$

Now what does this have to do with us?

In order to define a measure on a certain collection of subsets, this “certain collection of subsets” must have good properties. You haven’t forgotten, have you? When we partition the range of a function, there is a unique subset of the function’s domain whose function values, for the points inside it, lie in the range we want.

Look at the listed properties one more time. First, the whole domain must be in the sigma-algebra, because measures must be able to measure the whole domain we care about. Second, it is closed under complementation, because measures must be able to measure partitions of the domain. Third, it is closed under countable union, because measures must behave consistently on the union of countably many pieces of the domain. Suppose measures could measure a bunch of subsets of the domain but could not measure their union; wouldn’t that be ridiculous? What would happen to integration over parts of the domain?

So the appropriate domain of measures is sigma-algebras. This is where questions like “What is the probability of countably infinitely many points among the values of $X$ if $X$ has a Gaussian density?” become void for lack of subject matter.

The answer to such questions is that we have nothing to do with these kinds of subsets. They are not inside our sigma-algebra, and for that reason they are not measurable either!

Definition of Measure

Again, I’ll start with dry and precise mathematical definitions:

Definition Suppose $X$ is a set and $Σ$ is a sigma-algebra on $X$ . A function $μ$ from $Σ$ to $R \cup {- \infty, \infty}$ is called a measure if:

Its range includes only positive values and zero: $\forall E \subseteq X \Rightarrow μ (E) \geq 0$
Its output value for the empty set is zero: $μ (\emptyset) = 0$
It is countably additive on subsets of $X$ : $μ (\cup_{i = 0}^{\infty} E_{i}) = \sum_{i = 0}^{\infty} μ (E_{i})$

You see, once again the effect of the sigma-algebra can be felt. Especially in the third condition. If the sigma-algebra is not closed under countable unions, such a thing can no longer be defined as a condition for being a measure; because the result of the union will not necessarily be part of the domain for the measure to have a value for it, let alone for there to be a relation between this value and the sum of the measure-values of each of the sets.

If you paid attention in the previous paragraph, you noticed that I called the output value of a measure for a set its size. In fact, in Persian, “measure theory” is called the theory of sizes, which is not a bad name either; but in my opinion, “measure” is a more appropriate name for a function that “measures” the size of a set. Nevertheless, the output of the measure function is the size of the input set, and I will keep using this naming convention from here on as well.

Now let’s look at the conditions for being a measure more informally. First, the range of the function is only positive numbers, zero, and infinity. The reason for this condition should be clear from the intuition that was discussed for integration. The area of a rectangle, no matter how you measure its sides, cannot be negative! The second condition is likewise transparent. A set that has no members has size zero. If we return again to integration, if for an interval of values the value of the function does not fall in any subpart of the domain, then the area under the curve does not exist for that interval of values either:

But the third condition. If this consistency condition does not hold, the value of the integral will be different for different accuracies. You remember: we would partition the range of the function, make these partitions finer and finer, and its limit would become the value of the integral. If the size of the union of several sets is not the same as the sum of their sizes, then in principle we will have no guarantee that the limit exists. Even intuitively, human measurement also computes the union of two sets this way.

Partitioning — consistency with respect to union

So measures are not strange things. They are a mathematician’s precise observations about the way we deal with measurement. But the trait of a mathematician is that they try to abstract. That is, when they learn a rule, they use it in ten other places too. Measures are the same. Once we have a good definition of a measure, we can study problems whose measures we did not previously know.

Now it is time to examine an example of using this abstraction.

Example

Let’s take another look at computing expectation. We have a function like $f$ for which a probability distribution is defined on its domain. For example, the function $f$ can be the amount of damage caused by the presence of impurities in the raw materials for IC production. We also measure impurity in milligrams per liter. Suppose the probability of a particular amount of impurity in the raw materials has a Gaussian distribution (this is a completely reasonable assumption; see the central limit theorem.)

Now we want to know, on average, how much damage we will have in the produced ICs. To compute this value, we need to take the expectation:

l = 𝔼 f (X) = \int f (X) d P (X)

One way is numerical or formulaic computation. Another way is to create a number of samples from the distribution $P$ . For example, 100 of them: $X_{0}, \dots, X_{9} 9$ . Since these samples have been generated from the measure $P$ , their density is also proportional to that same measure. For this reason, we can estimate the amount of damage like this:

l = 𝔼 f (X) \approx \sum_{i = 0}^{99} f (X_{i})

In the problem of this example, computation by this method is not very attractive. But suppose we do not have the distribution of the amount of impurity in the raw materials, and only some measurements of impurities from previous days are available. Now the problem is different.

If you are familiar with the Kalman filter, I should say that there is a version of this filter for unknown distributions that works on this very basis, and it is called the particle filter.

The name of this integration method I introduced is also Monte Carlo integration.