Neural network not converging

Question

I'm new to Neural Networks, and programming generally. I've written a neural network in java, and i'm looking at football data. I have two inputs:

1) Home team win % over n games 2) Away team win % over n games

Using 'standard statistical models' one can predict the number of goals that will occur in a match using these two numbers alone, with a reasonable degree of accuracy. However, when i attempt to train my NN to predict the number of goals, it simply doesn't converge :(

I'm using a genetic algorithm to train the network, here is the fittest individual from the first few generations with a population size of 100,000:

1) 0.1407408056662556 2) 0.13406266176967252 3) 0.13406267600215235 4) 0.1338753567259805 5) 0.13280257001618265 6) 0.13275165964860766 7) 0.1319768652096691 8) 0.13161029326238236

Now i know it looks like it is converging, but it is converging at a painfully slow rate, and i have run this multiple times over many generations and it will not go below 0.13.

I am using a feedforward neural network, with one hidden layer of 10 neurons, and one output neuron. I am using a hyperbolic tangent sigmoid function in the hidden layers, and a sigmoid function for the output layer. I have divided the number of goals by 10, to give an output between 0 and 1.

Before i began running this, i assumed that the NN would outperform simple statistical models, but it doesn't come close. My question is:

From the results you can see, does it look like there is an error in the code somewhere ? Do i need to make a change to the architecture of the network ? Do i need to change the network inputs/training data in some way ?

I've been trying to identify the issue for a while now, and it's driving me mental. Any adive is greatly appreciated.

Many Thanks.

Answer 1

It might very well be that what you get is the best that could be achieved with such a neural network configuration. There are existing NN implementations, such as the multilayer perceptron in Weka. It would be better to start experimenting with those, where you have good analytical tools and confidence that the results you get are not due to an implementation error.