Non-parametric Bayes Decisions

If we assume that there is a model m generating the points of category X and another, f, generating the points of category O in ${\fam11\tenbbb R}^n$, then the above recipe can be applied providing we have estimates of the relative likelihoods of a new datum for each model, and prior probabilities for the models.

Using a neighbourhood and counting the fraction of the existing data which is of category X can be defended as giving an estimate of the likelihood function for model m in the neighbourhood. Taking the total number of points in each category as an estimate of the prior probabilities, we get that the likelihood ratio is simply the ratio of the different counts of the two categories in the neighbourhood. If the ratio stayed approximately the same as the size of the neighbourhood got smaller until the numbers were too small to inspire confidence in the estimate, then one might be inclined to buy this.

Similarly, we could find the closest k points to our new point and then count the fraction which are of category X to estimate the local pdf. The same kind of reasoning applies.

Observe that both of these methods presuppose that the metric is known and not negotiable. This may not be a convincing assumption. It is necessary to take a critical view of fundamental assumptions throughout this section: all too often there is something nasty hiding under the algebra.