Implementing Radial Basis Function Classifier in R

  • In this article, implementation of an RBF (Radial Basis Function) Classifier will be described for binary classification.
  • The algorithm consists of two different parts:
    1. Unsupervised: Find k << N centers (N = # data points) for the RBF network.
    2. Supervised: Compute the weights w of the classifier hypothesis h(x) from the (training) data using the labels.
  • k centroids will be used for the RBF network and those centroids will first be found with Lloyd’s algorithm.
  • Then parameter vector w will be computed by solving the least square equation which is linear in ϕ.
  • The following figure shows the outline of the algorithm to be used.

    algo1.png

 

  • Number of centers k and the the γ parameters will be varied to obtain different decision boundaries for classification.
  • As can be seen from the experiments on a few 2 dimensional (labeled) datasets, γ works as regularization parameter for the classifier, at lower value of the parameter, the classifier in-sample accuracy is higher.
  • The next two animations show the decision boundary contours learnt by the RBF classifier as a dark blue line on the same dataset for different γ values. The centers of the RBF network is represented by the blue stars surrounded by the black circles.
  • The first animation shows the results of the RBF classifier starting with 3 centers, while the second one starting with 5 centers.rbf3.gif
    rbf5.gifacc

 

  • The next two animations show the decision boundary contours learnt by the RBF classifier as a solid black line on another two different datasets, for different γ values. The centers of the RBF network is represented by the black stars surrounded by the black circles.
  • The first animation shows the results of the RBF classifier starting with 3 centers, while the second one starting with 5 centers.rbf3.1.gif
    rbf5.1.gif
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s