Theoretically a neural network can be trained
to calculate every countable function. Practically, neural networks show
best results in fields such as: classification, pattern recognition, voice
recognition, processing information with noise, approximation functions,
solving problems with a lot of available examples for training, and problems
which need fast selection role depending on input.
This is exactly the class of problems,
which is hard to handle with the usage of the common programming languages.
Exposing a neural net as the object component is the best way to utilize
advantages of this different way of solving problems.
 |
 |
 |
SOFTWARE
REQUEST FOR USING NEUROVCL COMPONENTS
|
1. Windows 95/98, NT/2000
2. Borland Delphi 5.0
3. NeuroVCL components
4. ANN V2.3 - neural net training program
|
|
|
ANN V2.3 - NEURAL
NET TRAINING PROGRAM
|
The program ANN is based on the backpropagation
neural networks. This program is used for training neural networks that
are used in programs developed by NeuroVCL components.
Figure 1. - ANN V2.3 - Neural network
training program
The program ANN is used to train neural
networks with patterns from training set. The following files are necessary
to assign neural network configurations and to determinate training set:
| Extension |
Description |
| *.INI |
This file describes neural network configuration:
number of layers, number of neurons on layers, learning coefficient, coefficient
to survey training. |
| *.INP |
Inputs – Training data set which
neural networks are trained to recognize |
| *.OUT |
Outputs - Training data set expected on
output. |
Files created by ANN after training the
neural network:
| Extension |
Description |
| *.NET |
This file is created with saving of all
important coefficients of the neural network. It contains the states of
all weights and thresholds of all the neurons in the network. |
| *.ERR |
This file contains recorded changes minimal,
maximum and average error at the end of each iteration. |
| *.LOG |
In this file is given values of neural
networks outputs and expected ones after XX iteration for all the patterns
from training set. |
| *.PAR |
In this file the weight and a threshold
change of every neuron of the network has been saved. |
After the successful training weights are
saved to file (with *.NET extension) and later are loaded in neuro components
which are used in the final application.
ANN is developed with the Delphi 5 programming
language. The program is developed with the usage of object-oriented methodology.
During the development, the following objects have been created: TArray,
TNeuron and TNet. These objects are used as a base for the development
of the NeuroVCL components for Delphi.
More informations ...
Free
Download!!! ANN V2.3 (ann.zip 463Kb)
|
|
|
OBJECT VCL COMPONENTS
FOR DELPHI
|
Classes' present a base for object oriented
programming. Class is the set of the related fields and methods (functions
and procedures), which are used to complete a certain program task. Classes
represent a set of data, and the objects are particular specimens of the
classes.
Programming in Delphi is based on VCL classes
(VCL Visual Components Class).
Free
Download (145KB)
http://ilicv.on.neobee.net/NEUROVCL.ZIP
With the help of TNNSignal type object,
the values of the neural network inputs are changed and read on the network
outputs after the calculation.
Look of components on form
Properties of TNNSignal:
| Properties name |
Type |
Description |
| Limited |
Boolean |
If this property is set True program does
not allow overvaluing min/max in property Value |
| Max |
Integer |
Maximum value of the signal if value is
limited |
| Min |
Integer |
Minimum value of the signal If value is
limited |
| Name |
String |
Object name |
| Value |
Real |
Value of signal |
Events of TNNSignal
OnChange - Events are performing if value
of property Value
is changed.
Methods
| Method name |
Description |
| Constructor Create |
Create object TNNSignal |
| Destructor Destroy |
Free memory taken by object |
The object of the TNeuralNet type presents
a neural network. This object does the processing of the inputs signals
(TNNSignal) and calculates appropriate values, which are read by output
signals (TNNSignal). Data from file (*.NET) with weights, generated by
ANN determinates the function of this object (performance, “algorithm”),
information on the way of outputs calculation, depending on inputs.
VCL
Net
Properties
| Properties name |
Type |
Description |
| Active |
Boolean |
Property determinates if network is active or not. |
| FileName |
TStrings |
Path to file (*.NET) that created after the neural network training
with ANN program. This file contains the states of threshold and weights
of the neurons of the network. |
| LayerHidden01 |
Integer |
Number of neurons on 1. hidden layer |
| LayerHidden02 |
Integer |
Number of neurons on 2. hidden layer |
| LayerHidden03 |
Integer |
Number of neurons on 3. hidden layer |
| LayerHidden04 |
Integer |
Number of neurons on 4. hidden layer |
| LayerHidden05 |
Integer |
Number of neurons on 5. hidden layer |
| LayerHidden06 |
Integer |
Number of neurons on 6. hidden layer |
| LayerHidden07 |
Integer |
Number of neurons on 7. hidden layer |
| LayerHidden08 |
Integer |
Number of neurons on 8. hidden layer |
| LayerHidden09 |
Integer |
Number of neurons on 9. hidden layer |
| LayerHidden10 |
Integer |
Number of neurons on 10. hidden layer |
| Name |
String |
Object name |
| Ni |
Real |
Learning coefficient |
| SignalInputs |
TStrings |
Names of input signals |
| SignalOutputs |
TStrings |
Names of output signals |
After placing the necessary objects in
to the form it is necessary to connect the neural network with input and
output signals. First we choose input signals by clicking on property SignalInputs.
The given dialog box is used for an easy selection of object names on a
neural network input. The same way we connect output signals with neural
network.
By connecting signals with neural networks
the value of property InputsLayer and OutuptsLayer is automatically changed.
Methods
| Method name |
Description |
| Constructor Create |
Creates object TNeuralNet |
| Destructor Destroy |
Free memory taken by object |
| Procedure LoadFromFile(FileName:String) |
Load weights and thresholds from file.
Note: Number of layers and neurons
on every layer in the program must be the same as the number of layers
and neurons on every layer in neural net what was trained by program ANN. |
| Procedure SaveToFile(FileName:String) |
Saves weights and thresholds to file. |
| Procedure Update |
Calculate values on networks outputs depending
on input data. |
|
|
|
EXAMPLE
1 - PROGRAM DEVELOPMENT USING NEUROVCL
|
Using ANN on created appropriate training
set is trained neural network for specific problem solving. Given weights
and thresholds from neural networks are saved to file, which is later used
in the program that is being created.
In this example neural network is trained
to recognize 12 different patterns size 3×3 (figure 1.1) in array size
5×5. These patterns can be placed in 9 positions in array (figure 1.2.).
The training set for this net is consisted of 108 elements (the number
of positions × the number of shapes) and the exact examples (figure 1.1,
and 1.2).
Neural network configuration:
Three layers neural net with backpropagation
algorithm of training:
Number neurons at input layer (number
of inputs): 5×5 = 25 (figure 1.1.)
Number neurons at hidden layer: 20
Number neurons at output layer (number
of outputs): 12
Learning coefficient (alpha): 0.25
Number of patterns in training set:
12×9 = 108 (figure 1.1 and figure 1.2)
Training set:
a) Inputs:
figure 1.1. - positions of object in array
ANN has been trained by using 3 textual
files (*.INI, *.INP, *.OUT). The details about file specification can be
read in [5, 6].
After a successful training the neural
network is necessary for saving weights and thresholds in the file (for
example shapes.net).
figure 1.2. - different objects
This
example shows the flow of the application building for the pattern recognition
in array 5×5. It is necessary to set 37 objects type of TSignal (25 inputs
+ 12 outputs). Set 25 TImage that reacts when user clicks on it and object
changes colors (black or white). By changing colors the value of the signals
(TSignal) linked to image (Algorithm 1.1.) were changed.
TForm1.Image1Click(Sender: TObject);
begin
If Image1.Canvas.Pixels[12,12]=RGB(0,0,0) then
begin
Image1.Canvas.Brush.Color:=RGB(255,255,255);
In01.Value:=0;
end
else
begin
Image1.Canvas.brush.Color:=RGB(0,0,0);
In01.Value:=1;
end;
Image1.Canvas.Rectangle(5,5,46,53);
end; |
Algorithm 1.1. Procedure links the
reaction of the user (clicking mouse button on image) with inputs of neural
networks.
The TneuralNet type object must be set
on the form. This object has to be linked with input and output signals
(figure 4).
Two buttons (TButton) are placed so that
the program could be controlled easier.
When designing of the user interface is
finished, it is necessary to add programming code to link events.
In procedure that is performed during the
start of the program, we should load network weights and thresholds and
activate objects.
Procedure TForm1.FormCreate(Sender: TObject);
Begin
NeuralNet1.LoadFromFile(shapes.net');
NeuralNet1.Activate:=true;
End; |
Algorithm 1.2 - Loading network coefficients
and activating object
Procedure that determinates what symbol
is drawn on array (Algorithm 1.3.).
Procedure TForm1.Button1Click(Sender: TObject);
Begin
NeuralNet1.Update;//update outputs on neural net
Max:=Out01.Value; //determinate max output value
Pos:=1;
If max<Out02.Value then
Begin
Max:=Out02.Value;
Pos:=2;
End;
.....
Repeated for every 12 values
.....
If pos=1 then label1.caption:='1';
If pos=2 then label1.caption:='2';
...
End;
Str(max:4:2,SMax);
Label2.Caption:=SMax;// probability
End; |
Algorithm 1.3 – Determination of the
symbol in array
Symbols that we want to determinate are
brought on input of neural network using classical programming method.
Comparisons of the values on neural network outputs are performed with
IF command, by which the symbol in the array is determined. As we can see
in this program neural networks can be viewed as complex function (black
box) with 25 inputs and 12 outputs.
|
|
|
EXAMPLE
2 - REALIZATON OF BYNARY FUNCTION ON NEURAL NETWORKS
|
Download
(source code) (23KB)
http://ilicv.on.neobee.net/neuro_examples.zip
Description of problem
Program demonstrate calculation of 10 binary
function with apliance neural networks.
In this program is used three layers neural
net with backpropagation algorithm of training:
Number neurons at input layer (number of
inputs): 2
Number neurons at hidden layer: 6
Number neurons at output layer (number
of outputs): 10
Learning coefficient (alpha): 0.25
Number of patterns in training set:
10
Training set which is used for neural network
training is shown in following table:
| input 1 |
input 2 |
output 1 |
output 2 |
output 3 |
output 4 |
output 5 |
output 6 |
output 7 |
output 8 |
output 9 |
output 10 |
| x |
y |
x and y |
x or y |
x xor y |
x => y |
x <=> y |
not(x) |
not(y) |
not
(x and y) |
not
(x or y) |
not
(x => y) |
| 0 |
0 |
0 |
0 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
0 |
| 0 |
1 |
0 |
1 |
1 |
1 |
0 |
1 |
0 |
1 |
0 |
0 |
| 1 |
0 |
0 |
1 |
1 |
0 |
0 |
0 |
1 |
1 |
0 |
1 |
| 1 |
1 |
1 |
1 |
0 |
1 |
1 |
0 |
0 |
0 |
0 |
0 |
Creating and training neural networks
First is created file BIN.INI that is used
to define neural network structure, number of layer and number of neurons
on every layer. Then is necessary to create two files to describe training
set (BIN.INP, BIN.OUT).
ANN (neural network training program)
is used to train created neural network and save states of connection weights
in file (BIN.NET).
Figure 2.1. - ANN - neural network training
program
Developing application in Delphi
On form was putted 12 object type TNNSignal
(2 for inputs and 10 for outputs of network) and 1 object type TNeuralNet.
In object inspector was adopted properties SignalInputs and SignalOutputs
by dialog box to conect input and output signals with neural network. On
hiden layer must be puted same number of neurons as in network during training
with ANN.
Figure 2.2. - Form in Delphi during development
application
When user click on button is executed following
procedure:
procedure TForm1.Button1Click(Sender: TObject);
var s:string;
begin
Input1.Value:=StrToFloat(Edit1.text); //set new value on
first network input
Input2.Value:=StrToFloat(Edit2.text); //set new value on
second network input
NeuralNet1.Update;
// count now values on network outputs
str(Output1.Value:2:2,s); // read value on first network
output and convert to string
Label1.Caption:=s;
// show value on first network output
str(Output2.Value:2:2,s);
Label2.Caption:=s;
str(Output3.Value:2:2,s);
Label3.Caption:=s;
str(Output4.Value:2:2,s);
Label4.Caption:=s;
str(Output5.Value:2:2,s);
Label5.Caption:=s;
str(Output6.Value:2:2,s);
Label6.Caption:=s;
str(Output7.Value:2:2,s);
Label7.Caption:=s;
str(Output8.Value:2:2,s);
Label8.Caption:=s;
str(Output9.Value:2:2,s);
Label9.Caption:=s;
str(Output10.Value:2:2,s);
Label10.Caption:=s;
end; |
Algoritam 2.1 - Učitavanje koeficijenata
mreže i aktiviranje objekta
Content of file with example N_SRC1.ZIP
/data
// copy files from this directory in directory data of program ANN
bin.ini
// information about network configuration
bin.inp
// traning set - inputs
bin.out
// traning set - outputs
/source
bin.net
// connection weights count using ANN V2.3
Project1.cfg
Project1.dof
Project1.dpr
Project1.res
Unit1.dcu
Unit1.dfm
Unit1.pas
Download
(source code) (23KB)
http://ilicv.on.neobee.net/neuro_examples.zip
|
|
|
EXAMPLE
3 - USING NEURAL NETWORKS FOR PATTERN RECOGNITION
|
Download
(source code) (23KB)
http://ilicv.on.neobee.net/neuro_examples.zip
Description of problem
This program demonstrate using of NeuroVCL
components for pattern recognition. Neural network is trained on examples
to recognize 8 shapes in matrix 3×3.
Za rešavanje problema upotrebljena je neuronska
mreža sa tri sloja sa backpropagation algoritmom obučavanja:
In this program is used three layers neural
net with backpropagation algorithm of training:
Number neurons at input layer (number of
inputs): 9
Number neurons at hidden layer: 6
Number neurons at output layer (number
of outputs): 8
Learning coefficient (alpha): 0.25
Number of patterns in training set:
8
Training set what is used for training
neural network. Training set is consisting eight patterns (figure 3.1).
Figure 3.1 - patrerns from training set
Neural network have 8 outputs (every output
present one of patterns). Output of neural networks is one value 1 and
rest of values is 0. Position of 1 determinate patrern on input.
Creating and training neural networks
First is created file PATTERNS.INI that
is used to define neural network structure, number of layer and number
of neurons on every layer. Then is necessary to create two files to describe
training set (PATERNS.INP, PATERNS.OUT).
ANN (neural network training program)
is used to train created neural network and save states of connection weights
in file (PATERNS.NET).
Figure 3.2 - ANN - neural network training
program
Developing application in Delphi
On form was putted 17 object type TNNSignal
(9 for inputs and 8 for outputs of network) and 1 object type TNeuralNet.
In object inspector was adopted properties SignalInputs and SignalOutputs
by dialog box to conect input and output signals with neural network. On
hiden layer must be puted same number of neurons as in network during training
with ANN.
Figure 3.2 - Form in Delphi during development
application
procedure TForm1.Image1Click(Sender: TObject);
begin
if image1.Canvas.Pixels[12,12]=RGB(0,0,0) then
begin
image1.Canvas.brush.Color:=RGB(255,255,255);
In01.Value:=0;
// <- field color white
end
else
begin
image1.Canvas.brush.Color:=RGB(0,0,0);
In01.Value:=1;
// <- field color black
end;
image1.Canvas.Rectangle(5,5,46,53);
end; |
Algoritam 3.1 - conecting user reaction
(click on Image) with neural network
procedure TForm1.Button1Click(Sender: TObject);
var max:real;
pos:integer;
smax:string;
begin
NeuralNet1.Update;
// <- update values on neural network output
max:=Out01.Value;
pos:=1;
if max<Out02.Value then
begin max:=Out02.Value; pos:=2;
end;
if max<Out03.Value then
begin max:=Out03.Value; pos:=3;
end;
if max<Out04.Value then
begin max:=Out04.Value; pos:=4;
end;
if max<Out05.Value then
begin max:=Out05.Value; pos:=5;
end;
if max<Out06.Value then
begin max:=Out06.Value; pos:=6;
end;
if max<Out07.Value then
begin max:=Out07.Value; pos:=7;
end;
if max<Out08.Value then
begin max:=Out08.Value; pos:=8;
end;
Case pos of
1:label9.caption:='1';
2:label9.caption:='2';
3:label9.caption:='3';
4:label9.caption:='4';
5:label9.caption:='5';
6:label9.caption:='6';
7:label9.caption:='7';
8:label9.caption:='8';
end;
str(max:4:2,smax);
label10.caption:=smax;
end; |
Algoritam 3.2 - updating network outputs
and showing results
Content of file with example N_SRC2.ZIP
/data
// copy files from this directory in directory data of program ANN
paterns.ini // information about
network configuration
paterns.inp // traning set - inputs
paterns.out // traning set - outputs
/source
paterns.net // connection weights
count using ANN V2.3
Project1.cfg
Project1.dof
Project1.dpr
Project1.res
Unit1.dcu
Unit1.dfm
Unit1.pas
Download
(source code) (23KB)
http://ilicv.on.neobee.net/neuro_examples.zip
NeuroVCL components in combination with
ANN
- neural network training program is not limited on patterns recognition.
These components can be used for solving any problem, which can be presented
in an appropriate way.
With the example given in this text we
have seen that neural networks and classical methods for solving the problems
(programming languages) are interlarded. The NeuroVCL components can be
observed as an expansion of possibilities of Delphi programming languages.
The advantage of such combining is that
the programmer does not have to know complex mathematics and logic functions
that link inputs and outputs, as he must know that he solved the problem
by classic method.
With the appliance of the object oriented
programming, neural network is presented as an object, which is simply
used for solving the exact problem. With such a method the programmer is
free from necessity of knowing inputs and outputs of the inside projection
principles of the neural network. It can be observed as a black box (unknown
complex logic function).
The advantage of using the object components
regarding to the other software that makes the appliance of the neural
networks possible, is that the programmer may write an application of his
own, not depending on other software (exp. MathLab).
Also, with the appliance of the object
components, the programmer has freedom for creating the users interface,
what differs from the specialized application.
Using ANN and objects,
which are forgoing developing NeuroVCL components, is developed
OCR
V1.1 program for recognizing Cyrillic letters [5, 6].
Free
Download (145KB)
http://ilicv.on.neobee.net/NEUROVCL.ZIP
Please send me a comments, suggestion and
critics about improvement NeuroVCL components in newer version. Also if
you have interesting examples with NeuroVCL components please send me on
mail ilicv@EUnet.yu.
[1] “Frequently asked questions about AI”,
http://www.cs.cmu.edu/Web/Groups/AI/html/faqs/ai/ai_general/top.html
[2] “Neural Network Frequently asked questions”,
ftp://ftp.sas.com/pub/neural/FAQ.html
[3] Hotomski, P., (1995): “Sistemi Veštačke
inteligencije”, Tehnički fakultet “Mihajlo Pupin”, Zrenjanin
[4] Ilić, V., (1999) “ANN - Program za
obučavanje neuronskih mreža sa backpropagation algoritmom učenja”, http://SOLAIR.EUnet.yu/~ilicv/ANN.html
[5] Ilić, V., (1999) “Obučavanje neuronskih
mreža za prepoznavanje ćiriličnih slova”, magistarski rad, Tehnički Fakultet
“Mihajlo Pupin”, Zrenjanin
[6] Ilić, V., (1999) “OCR - Program za
prepoznavanje ćiriličnih slova”, http://SOLAIR.EUnet.yu/~ilicv/ocr.html
[7] Jocković, M., Ognjanović Z., Stankovski
S. (1997) “Veštačka inteligencija inteligentne mašine i sistemi”, Grafomed,
Beograd
[8] Milenković, S., (1997): “Veštačke
neuronske mreže”, Zadužbina Andrejević, Beograd
[9] Reisdorph, K., (1999): “Naučite Delphi
4 za 21dan”, Kompjuter biblioteka, Čačak
[10] Sajić, I., (1995): “Neuronske mreže”
, časopis “Računari” br 108, BIGZ, Beograd
[11] Savić, D., (1992), “Uvod u objektno
programiranje”, PC Program, Beograd
[12] Subašić, P., (1998): “Fazi logika
i neuronske mreže”, Tehnička Knjiga, Beograd
