Distance-Weighted Nearest Neighbor Algorithm

• The refinement to the k-NEAREST NEIGHBOR Algorithm is to weight the contribution of each of the k neighbors according to their distance to the query point xq, giving greater weight to closer neighbors.
• For example, in the k-Nearest Neighbor algorithm, which approximates discrete-valued target functions, we might weight the vote of each neighbor according to the inverse square of its distance from xq

 Distance-Weighted Nearest Neighbor Algorithm for approximation a discrete-valued target functions

Distance-Weighted Nearest Neighbor Algorithm for approximation a Real-valued target functions

 

Terminology

• Regression means approximating a real-valued target function.
• Residual is the error 𝑓̂(x) - f (x) in approximating the target function.
• Kernel function is the function of distance that is used to determine the weight of each training example. In other words, the kernel function is the function K such that

wi = K(d(xi, xq))