At the forefront of Artificial Intelligence

Mathematics for AI Beginners

There have been a lot of requests for a mathematical introduction for those of you less acquainted with mathematical notation and their meanings. This essay will introduce the necessary mathematics to understand a variety of formulas within neural networking and genetic algorithms. It will not cover the theory because it is simply too broad.

Neural Network Examples

Let us look at a very simple equation - the Perceptron Learning Rule:

Simple equation

Firstly, the triangle denote the change or "delta" (thus the Greek letter delta). 'w' is an set of numbers and i is used to denote the index of the set. So in english, the above equation would read:

"The change of the i^th member of w is equal to the i^th member of x multiplied by d"

Ok, let us look at something quite a bit more complicated. Here is the equation that governs weight modification in back-propagation:

Scary? It shouldn't be - mathematical equations are almost never as complex to understand as they look. That funny shaped "E" stands for summation. Here p is used as a counter variable and runs from 0 to P. So here the equation states:

The change of the i^th member of w is equal to n divided by P multiplied by the sum x_ipd_p for all p's between 1 and P.

Some Exercises

Those 3 concepts (change, sets and summation) should get you through a lot of rudimentary AI mathematics (at least understand how to compute things from their formulas). Here are some exercises to make sure you understand. Try to equate the outcome of these three exercises:

(Where W is a set of all odd numbers)
(Where E is a set of all even numbers, calculate for i=1...5)

Ok and here are the answers!

1 + 2 + 3 + 4 + 5 = 15
1 + 3 + 5 + 7 + 9 + 11 + 13 + 15 + 17 + 19 = 100
For i₁ = 4 + 8 + 12 + 16 + 20 + 24 + 28 + 32 + 36 + 40 = 220
For i₂ = 8 + 12 + 16 + 20 + 24 + 28 + 32 + 36 + 40 = 216
For i₃ = 12 + 16 + 20 + 24 + 28 + 32 + 36 + 40 = 208
For i₄ = 16 + 20 + 24 + 28 + 32 + 36 + 40 = 196
For i₅ = 20 + 24 + 28 + 32 + 36 + 40 = 180

Conclusion

Mathematical symbols can look very daunting, but more often than not are just a concise way of representing something long but relatively simple to accomplish. If you are the type that thinks more in computational terms (like myself) write a program or jot down on a piece of paper the necessary steps. Look at code that implements these things (such as BP code) - it will all look a lot simpler. If you feel another mathematical concept should be included here, please mail me.

Last Updated: 21/10/2001

Article Toolbar

BibTeX entry