Histograms and Probability Density Functions

If there are only a finite number of possible things we might have as outcomes, we can enumerate them, 1 to k, and, if the random variable is really given to us explicitly, compute the measure for each of them. We could draw a histogram of outputs and probabilities associated with them. Such a thing is called a probability distribution and we are all familiar with them. For a coin which can only come down Heads or Tails, we can let 0 represent Tails, 1 Heads, and erect a little pole of height 0.5 over each of these two numbers.

 

Figure 3.2: Histogram/PDF for a RV taking values in ${\fam11\tenbbb R}$.

If the random variable takes a continuum of values, as when someone hurls darts at a board, we cannot assign a probability to any single outcome, since under normal circumstances this will be zero. But we can still draw a histogram for any choice of boxes partitioning the space, ${\fam11\tenbbb R}^n$, of values of the rv. If we normalise, so that the area, volume or in general measure under the histogram is one, and then do it again with a finer partition, we can get closer to a continuous distribution. In the limit, with some technical conditions satisfied that you can safely ignore because they are no more than mathematical book-keeping, you may get a continuous non-negative function over ${\fam11\tenbbb R}^n$, with integral 1, and this is known as a probability density function, pdf for short, and again they are exceedingly familiar. There are some niceties; the pdf may not be continuous and the values may be mixed discrete and continuous, but we need not contemplate these issues in our applications. The usual derivation of the pdf from the measure is different from the present hint that you do it by limits of histograms, and has little to recommend it unless you are an analyst.

It is worth noting that since we can only measure vectors to some finite precision and we absolutely never get a data set which is of uncountably infinite cardinality, the distinction between a very fine histogram and a real pdf is somewhat metaphysical. It is also worth noting that a hypothetical measure space and map from it, about which nothing can be said except the relative meaures of bits corresponding to sets of outputs from the map, is also a touch metaphysical. If you can use one model for the rv and I can use another with a different domain space, and if we agree on every calculation about what to expect, then it must cross one's mind that we might manage to do very well without having an rv at all. All we really need is a measure on the space of outcomes, the sample space. If we reflect that to specify a measure on a sample space which is some region in ${\fam11\tenbbb R}^n$ the simplest way is by giving a density function, we see that it might have been simpler to start with thepdf.