Baysig quick tour: Probability distributions

This is the second document in a quick tour of the Baysig programming language, following on from Baysig quick tour: Fundamentals.

Baysig has one elementary probability distribution built-in (unit) and many more defined in its library. We can plot the histogram approximately corresponding to the probability density functions for each distribution.

Further probability distributions can be built from this elementary one. Some are already defined in Baysig's standard library, such as the standard normal distribution unormal:

a uniform distribution with lower and upper bounds uniform 5.0 10.0

and the gamma distribution, here gamma 3 2

The real power of Baysig lies in the possibility of composing probability distributions. This is done by drawing a sample from one distribution, which can then be used as a parameter in another probability distribution. For instance, we could draw the mean of a normal distribution from a uniform distribution:

uniMean = prob
  mean ~ uniform 1 5
  normal mean 5

Now that uniMean has been defined, it too can be used to define further probability distributions.

uniMeanExp n = prob
   x ~ uniMean
   return (x^n)

This example illustrates a few new things: probability distributions can take arguments, just like functions. In fact, distributions are functions and are subject to the same flexibility: they can be stored in variables and passed as arguments to other functions or distributions. It may look as if these definitions introduce a special syntax which deviates from the functions, patterns and data types paradigm we introduced in the first section. And how to reconcile probabilistic programming with purely functional programming, in which a function applied to the same inputs must always give the same outputs? The last section in this tour demonstrates how probabilistic non-determinism can be elegantly wrapped up in purely functional programs and give a clear meaning to the notation introduced above.

uniMeanExp also uses the return function, which turns a deterministic expression into a probability distribution. It is most useful when expressing some function or combination of previously sampled values.

Can we see individual values drawn from a probability distribution? If we ask for the distribution to be displayed (more on how to do this in the section on reproducible research), Baysig will display the mean and standard deviation of the generated values.

uniMeanExp 2.0 ⇒ 14 ± 16

But we might also ask for a single value sampled from the generator to use in further calculation. In a sense, this is what happens inside a do-block. In x ~ D, x behaves like a particular value drawn from the distribution D. But the value of x can never "escape" from the distribution. This is an important point, because distributions in a sense represent uncertainty which must never be collapsed into a particular value. So when we use the operator sample to draw values from distributions, it is strictly speaking cheating. To prevent misuse of sample, it is not implemented as a function and can only be used at the top level (i.e. not inside a definition such as a function or a probability distribution). We also do not use equality to denote this assignment:

u <* sample (uniMeanExp 2.0)

u  ⇒ 7.1936

Similar syntax (<*) is used for other operations that are not purely functional, for instance loading data from disk.

One sample doesn't really tell much. The Baysig library has a number of utility functions for manipulating probability distributions. One such function is repeat which creates a new probability distribution over a list containing a integer number of draws from an existing distribution:

us <* sample (repeat 5 (uniMeanExp 2.0))

us ⇒ [5.0183, 5.2700, 19.5887, 3.1935, 4.5574]

The quick tour of Baysig continues in the document Baysig quick tour: Bayesian statistical inference