At the forefront of Artificial Intelligence

Home » Articles » Neural Networks » Beginner

Simple OCR Using Perceptrons

This article will look at using perceptrons for something slightly beyond the trivial - recognizing noisy version of the digits 0-9 as demonstrated by ONR below. Please note it is assumed you understand the theory of perceptrons, read the Generation5 essay if you are unsure.

Understanding the Problem

How can this problem be seen as a classification problem? Put simply, we want to classify what is-number-x and what isn't-number-x. Since we can only differentiate between what is the correct digit and what isn't, we will need ten perceptrons—one for each of the digits 0-9.

To create the learning data, we want to present each perceptron with its own digit as the correct sample, and all the other digits as the incorrect samples. This is insufficient however, since we need the perceptrons to be able to recognize noisy digit data too. Therefore, we need to generate noisy versions of each of the digits, and tell each perceptron to accept the clean and noisy versions of its assigned digit and disregard all the others. For example, ONR typically generates 25 templates per digit, with one clean template and twenty-four with 5% noise added:

This means each perceptron will have 25 templates that it must learn to classify as is-a-digit and 225 to classify as not-a-digit.

Implementation - ONR

Example implementation of this system uses Java and the Generation5 JDK. While all the perceptron code is held in org.generation5.ai.PerceptronNN, all the GUI and data-handling code is in onr.OnrFrame. ONR handles the data by generating an initial set of training data (generateData), then trains the perceptrons by creating perceptron-specific training data (createDigitData). createDigitData copies all the correct digits to the training set's class-B data array, and all false data to class-A.

With this in mind, let us look at the perceptron training function:

public void trainPerceptrons() {
    for (int i=0; i<perceptron.length; i++) {
        // Create the perceptron with 35 inputs
        perceptron[i] = new PerceptronNN(35);
            
        int cc = 0;  // number correctly classified

        // Create perceptron-specific training data
        PerceptronData pd = createDigitData(i);
        for (int a=0; a<100; a++) {
            cc = (int)perceptron[i].train(pd);
            if (cc == trainingData.length) break;
        }
    }
}

You can see how we create the perceptron with 35 weights (although an extra is automatically created to act as a bias), create the perceptron-specific data, and then attempt to train the perceptron for up to 100 iterations.

Perceptron Output

Now that the system has been trained using the data, how do we recognize an unseen character? What is the output from the perceptrons? Since perceptrons are taught to classify between class A and class B, the output will be 0 for non-digit and 1 for the digit. Therefore, we simply have to iterate through each perceptron and present it with the input data. The perceptron that 'fires' (returns a 1) should hopefully be our digit (see classifyButtonActionPerformed for implementation).

Using ONR

Assuming generation5.jar lies somewhere in your CLASSPATH, then running ONR is simple as double-clicking on ONR.jar or running it on the command-line:

java -jar onr.jar

The perceptrons are trained at load-time, and the test data is automatically written to testData.png in the current directory. To test the perceptrons, click on one of the digit templates to load the digit into the input panel (left). Click 'Add Noise' a couple of times to distort the input image and then click 'Classify' to see if the digit display in the output panel displays the correct digit.

onr.zip (18Kb)

Submitted: 19/08/2004

Article Toolbar

BibTeX entry