At the forefront of Artificial Intelligence

Home » Articles » Neural Networks » Applications/Code

Simple BMP File Analysis Using MLPs

By Povilas Daniuðis

pdaniusis@ik.su.lt

Abstract

MLP (multilayer perceptron) is great method to perform clasification tasks. However "real world" clasification problems are very noisly and high-dimensional.So to apply MLP you must solve how to extract as many as possible information from raw input data.In this text we will describe little experiment with MLP.We'll use not real world,but generated (human generated,so - little noisly) data - BMP (bitmap) files.

Introduction

In this text we will discuss one not complex but quite interesting MLP application.In BMP file we have little man face.We'll try to teach MLP to determine he smiles or not.For example:

To do this we'll need:

MLP software
Raw training data
Raw validation data
Method how to extract MLP - friendly training data from raw training data.

Note: We will not discus what MLP is,how it works etc.We'll assume,that reader is familiar with MLP.

MLP software

For obviuos reasons,i prefer my own MLP software,perceptron training system Petras,but of course you can use any MLP trainer/simulator. Petras is written in C++,it uses some optimizations like dynamic weight change,momentum etc(learning alghoritm is back-propagation, about what you can read in generation5 essays or many where on the web). You can download this program from my web site.

Raw training data and validation data

This was my first experiment with neural networks,so face after face i painted them in my own hands ;).To reach accaptable MSE (mean squared error), we will need about 300 faces.

Method how to extract MLP - friendly training data

Each BMP file of raw training data is 122 bytes long.This is quite big input vector and we will need about 1000 examples to reach acceptable MSE.So wee will use simple method how to reduce input vector size to 27 and avoid pleasure of painting 1000 or more faces.

Grab 9 lowest lines.Each line represents 4 bytes.But 4-th byte always was same,so we simply do not include it.So we have 3x9=27 bytes.
Adding to training data some random noise greatly improves results.In this experiment we will add random number from 0 to 6 to each component of input vector.

Here is source code to do all this : /* This program generates smiles.ann from bitmap files. File smiles.ann is training data for perceptron training system Petras. Author: Povilas Daniushis. */ #include <stdio.h> #include <stdlib.h> #include <memory.h> #include <time.h> #define HI 0.9 #define LO 0.1 #define PATERNS 321 //how many training patterns int main() { FILE *fp; FILE *f; char out[128]; char in[128]; char c; int i,j,b; int nl=0,vn=0; int targs[] = {1,1,1,0,1,0,1,0,0,1, //desired targets 1,0,1,1,1,0,1,0,1,0, 0,0,1,1,0,0,0,1,1,0, 1,1,1,1,1,1,0,0,1,0, 1,0,1,0,1,1,1,1,1,1, 1,1,0,0,0,0,0,1,0,1, 0,0,1,0,1,0,0,1,0,0, 1,0,1,1,0,0,1,1,0,1, 0,1,0,0,1,1,0,1,1,1, 1,0,0,0,1,0,1,1,1,0, 1,1,1,0,1,1,0,0,1,1, 1,1,0,0,0,0,0,0,0,1, 0,0,1,1,1,1,1,1,1,1, 1,0,0,0,0,1,0,1,0,0, 0,1,0,0,0,0,0,0,1,0, 1,1,0,1,0,1,0,0,1,1, 0,1,0,1,1,0,1,0,0,0, 0,1,0,0,0,1,0,0,0,0, 0,0,1,0,1,1,0,0,0,0, 1,0,0,0,0,1,1,1,0,1, 0,1,0,1,1,1,0,1,0,1, 0,0,1,1,0,0,1,0,1,1, 0,1,1,0,1,0,0,1,0,1, 1,0,1,1,0,1,0,1,0,1, 0,0,1,1,0,1,0,1,1,1, 0,0,0,1,0,1,0,0,0,1, 1,1,0,1,1,1,0,1,1,1, 1,0,1,0,0,1,0,1,0,1, 0,0,0,1,0,1,1,1,0,1, 0,1,0,1,0,1,0,0,0,0, 1,0,0,1,0,1,1,0,1,1, 1,1,0,0,1,0,1,0,0,1, 1}; printf("Generating ANN script : \n"); if ((f = fopen("smiles.ann","w")) == NULL) exit(printf("Error opening smiles.ann for writing . \n")); //parameters of neural network 4 petras : fprintf(f,"0\n3\n27 30 1\n0.25\n0.01\n10000\n%d\n",PATERNS); for (i = 0; i<PATERNS; i++) { memset(out,0,sizeof(out)); sprintf(out,"%d",i+1); strcat(out,".bmp"); vn+=targs[i]; printf("Reading %s (1 %d 0 %d)\n",out,vn,i-vn); fp = fopen(out,"r"); if (fp == NULL) exit(printf("I/O error \n")); fprintf(f,"\n\n"); j = 0; b = 0; while (!feof(fp)) { j++; fscanf(fp,"%c",&c); if (j>62 && j<62+4*9) // 4 lowest lines { if ((j - 62)%4) //if not 4-th element { fprintf(f,"%d ",c + rand()%7); //add some random noise. b++; } } } printf("%d bytes\n",b); if (targs[i] == 1) fprintf(f,"\n %f \n",HI); else fprintf(f,"\n %f \n",LO); fclose(fp); } printf("Complete.\n"); fclose(f); }

Just compile it,place executable in folder with raw training data 1.bmp,2.bmp,...,321.bmp and execute.You will get training data for my perceptron training system Petras.

Neural network parameters

We do not need negative outputs,so as activation function was used logistic sigmoid.

In this experiment we use 3 layer MLP with 10-60 units in one hidden layer.Best results was produced by 30 unit hidden layer networks. In output layer we have one neuron,what returns net output: 0.9 if MLP thinks it is smile and 0.1 if non smile.
Note: We could change 0.9 to 1 and 0.1 to 0,but then we will have small problem - to reach for example >> 1 we must have big NET sum (ant for >> 0 - we must have big negative net sum),so, big weights and slow learning (see back-propagation alghoritm for details).

Momentum factor = 0.9,initial learning rate = 0.25.

Results

After training MLP provides quite nice results.There is example for validation sub-set:

Sub-set α (non atypical data)
0.991024

0.085072

0.988719

0.114802

0.763495

0.425314

0.975805

0.053915

0.999819

0.082987

0.995026

Sub-set β(atypical data)
0.030439

0.192780

0.120018

0.037295

0.654652

0.140132

0.916232

0.256333

0.034709

0.003876

0.969863