php-fann | PHP wrapper for FANN ( Fast Artificial Neural Network Library | Artificial Intelligence library

Long story short, your dataset is likely too complex for such a small and simple network.

When I wrote the OCR example, and I was kind of showing off a little by "compressing" all 94 chars into a single output neuron. It's not typically done this way and certainly not with complex datasets.

Usually, you would want to dedicate an output neuron for each "class" that the network needs to identify.

Put simply, its harder for the network to learn to properly increment or decrement the output value by 0.01 on a single neuron (as is the case of my OCR ANN) than to learn to associate a dedicated output neuron / pattern with a specific class.

You can find a better example of a more typical classifier implementation in the MNIST subfolder in my repo for the OCR "family" of neural networks: https://github.com/geekgirljoy/OCR_Neural_Network

My suggestion is to redesign your ANN.

Based on your code your network looks like this:

L0: IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

L1: HHHHHHHHHHHHHHHH

L2: O


Whereas it would probably operate (classify) your data better if you redesigned it like this:

First, determine the number of distinct classes types, in the example you gave I saw 0.07 listed so I will assume there are seven different classes of order types.

So, the ANN should look like this:

L0: IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

L1: A sufficient number of "hidden" neurons

L2: OOOOOOO

Where O1 represents class 1, O2 class 2 etc...

Which means that your training data would change to something like this:


60000 32 7
0 0 0 0 0 1 1 0 0 1 1 0 1 1 0 0 1 1 1 0 1 1 1 0 0 0 1 1 0 0 1 0
1 0 0 0 0 0 0
0 0 0 0 0 1 1 0 0 1 1 0 1 1 0 0 1 1 1 0 1 1 1 0 0 0 1 1 0 1 0 0
1 0 0 0 0 0 0
0 0 0 0 0 1 1 0 0 1 1 0 1 1 0 0 1 1 1 0 1 1 1 0 0 0 1 1 0 1 1 0
1 0 0 0 0 0 0
0 0 0 0 0 1 1 0 0 1 1 0 1 1 0 0 1 1 1 0 1 1 1 0 0 0 1 1 1 0 0 0
1 0 0 0 0 0 0
0 0 0 1 1 1 0 1 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 0 1 0
0 0 0 0 0 0 1
0 0 0 1 1 1 0 1 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 1 0 0
0 0 0 0 0 0 1
0 0 0 1 1 1 0 1 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0
0 0 0 0 0 0 1
0 0 0 1 1 1 0 1 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 0 1 0 1 0 0 0 0
0 0 0 0 0 0 1

Class Output Examples:

Class 1: 1 0 0 0 0 0 0
Class 2: 0 1 0 0 0 0 0
Class 3: 0 0 1 0 0 0 0
Class 4: 0 0 0 1 0 0 0
Class 5: 0 0 0 0 1 0 0
Class 6: 0 0 0 0 0 1 0
Class 7: 0 0 0 0 0 0 1


Also, depending on your methodology, you MAY get better results using a harder negative value like -1 instead of 0, like this:

60000 32 7
-1 -1 -1 -1 -1 1 1 -1 -1 1 1 -1 1 1 -1 -1 1 1 1 -1 1 1 1 -1 -1 -1 1 1 -1 -1 1 -1
1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 1 1 -1 -1 1 1 -1 1 1 -1 -1 1 1 1 -1 1 1 1 -1 -1 -1 1 1 -1 1 -1 -1
1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 1 1 -1 -1 1 1 -1 1 1 -1 -1 1 1 1 -1 1 1 1 -1 -1 -1 1 1 -1 1 1 -1
1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 1 1 -1 -1 1 1 -1 1 1 -1 -1 1 1 1 -1 1 1 1 -1 -1 -1 1 1 1 -1 -1 -1
1 -1 -1 -1 -1 -1 -1
-1 -1 -1 1 1 1 -1 1 1 1 1 -1 -1 1 -1 -1 -1 1 -1 -1 1 1 1 -1 -1 1 -1 -1 1 -1 1 -1
-1 -1 -1 -1 -1 -1 1
-1 -1 -1 1 1 1 -1 1 1 1 1 -1 -1 1 -1 -1 -1 1 -1 -1 1 1 1 -1 -1 1 -1 -1 1 1 -1 -1
-1 -1 -1 -1 -1 -1 1
-1 -1 -1 1 1 1 -1 1 1 1 1 -1 -1 1 -1 -1 -1 1 -1 -1 1 1 1 -1 -1 1 -1 -1 1 1 1 -1
-1 -1 -1 -1 -1 -1 1
-1 -1 -1 1 1 1 -1 1 1 1 1 -1 -1 1 -1 -1 -1 1 -1 -1 1 1 1 -1 -1 1 -1 1 -1 -1 -1 -1
-1 -1 -1 -1 -1 -1 1


This is because you are using a "symmetric" hidden/output function like FANN_SIGMOID_SYMMETRIC which is a sigmoid and so the relationship between -1 to 0 and from 0 to 1 isn't linear so you should get better/harder distinctions between classifications and potentially faster training / fewer training epochs by more strongly contrasting the inputs/outputs like this.

Anyway, once you have trained the network and run your tests, you simply take the max() output neuron as your answer.

Example:

// ANN calc inputs and store outputs in the result array
$result = fann_run($ann, $input);

// Lets say the ANN responds like this:
// [-0.9,0.1,-0.2,0.4,0.1,0.5,0.6,0.99,-0.6,0.4]

// Let's also say there are 10 outputs representing that many classes
// 0 - 9
// [0,1,2,3,4,5,6,7,8,9]
//
// Find which output contains the highest value (the prediction/classification)
$highest = max($result); // $highest now contains the value 0.99

// So to convert the highest value to a class we find the key/position in the $result array
$class = array_search($highest, $result);

var_dump($class);
// int(7)

Why? Because the 7th key (7th / 8th (depending on how you look at it) is the high value):

array(0=>0.9,
1=>0.1,
2=>-0.2,
3=>0.4,
4=>0.1,
5=>0.5,
6=>0.6,
7=>0.99,
8=>-0.6,
0=>0.4
);

In the case of multiple class types being possible at the same time, you "softmax" instead.

Hope this helps! :-)