Welcome to Reproducing Kernel Hilbert Space
In a series of posts I hope to introduce Mathemafrica readers to some useful data analysis methods which rely on operations in a little back-water of Hilbert space, namely Reproducing Kernel Hilbert Space (or RKHS).
We’ll start with the “classic” example. Consider the data plotted in figure 1. Each data point has 3 “properties”: an 
Figure 1: A scatter of data with three properties: an x_1 coordinate, an x_2 coordinate and a colour.
Suppose for each data point we generate a representation of the data point ![\phi(x)=[x_1, x_2, x_1x_2]](http://s0.wp.com/latex.php?latex=%5Cphi%28x%29%3D%5Bx_1%2C+x_2%2C+x_1x_2%5D+&bg=ffffff&fg=000&s=0 "\phi(x)=[x_1, x_2, x_1x_2]")
![p\in[0~ 1]](http://s0.wp.com/latex.php?latex=p%5Cin%5B0%7E+1%5D+&bg=ffffff&fg=000&s=0 "p\in[0~ 1]")
, associated with a specific possible outcome of an observable occurrence or process, is simply telling you that, could you observe this occurrence (or process) infinitely many times, the fraction of such observations that would yield that specific outcome is 
. Using the age-old coin toss example: tossing the coin is the occurrence or process and recording a Heads or Tails are the two observation. The number 0.5 
tells a Frequentist that, in the pursuit of infinitely many coin tosses, the ratio of Heads recorded to the number of tosses performed asymptotically approaches 0.5. And that’s all! The value should not be interpreted as the most likely outcome for the next observation or sample taken from the process (though I’ve always wondered how a Frequentist would gamble…).…