LSTM: Figuring out the library?

问题:

I'm using the library https://github.com/cazala/synaptic

I am trying to predict the next value (value X) in the following series:

0 0 0 1 0 0 0 1 0 0 0 X

Which should be a 1.

Here is the code:

const options = {
    peepholes: Layer.connectionType.ALL_TO_ALL,
    hiddenToHidden: false,
    outputToHidden: false,
    outputToGates: false,
    inputToOutput: true,
};
// 1 input, 3 hidden layers (4 nodes per layer), 1 output
const lstm = new Architect.LSTM(1, 4, 4, 4, 1, options);
const trainingArray = [
    {
        input: [0],
        output: [0],
    },
    {
        input: [0],
        output: [0],
    },
    {
        input: [0],
        output: [1],
    },
    {
        input: [1],
        output: [0],
    },
    {
        input: [0],
        output: [0],
    },
    {
        input: [0],
        output: [0],
    },
    {
        input: [0],
        output: [1],
    },
    {
        input: [1],
        output: [0],
    },
];
const trainingOptions = {
    rate: 0.1,
    iterations: 100000,
    error: 0.05,
    cost: null,
    crossValidate: null,
};
let results = lstm.trainer.train(trainingArray, trainingOptions);
console.log(results);
array = [
    0,
    0,
    0,
    1,
    0,
    0,
    0,
    1,
    0,
    0,
    0,
];
results = lstm.activate(array);
console.log(results);

The output in the console:

{ error: 0.049765018466871494, iterations: 673, time: 392 }
[ 0.05010961302724895 ]

I was expecting the result of the activation to be a value closer to 1 than 0 (much closer). I don't know if this is the library of my lack of knowledge with LSTM. Can someone point me in the correct direction?

回答1:

I read through the source code and figured it out.

const synaptic = require('synaptic');

const Architect = synaptic.Architect;
const Layer = synaptic.Layer;

const lstmOptions = {
    peepholes: Layer.connectionType.ALL_TO_ALL,
    hiddenToHidden: false,
    outputToHidden: false,
    outputToGates: false,
    inputToOutput: true,
};
const lstm = new Architect.LSTM(1, 4, 4, 4, 1, lstmOptions);

const trainSet = [
    { input: [0], output: [0.1] },
    { input: [1], output: [0.2] },
    { input: [0], output: [0.3] },
    { input: [1], output: [0.4] },
    { input: [0], output: [0.5] },
];
const trainOptions = {
    rate: 0.2,
    iterations: 10000,
    error: 0.005,
    cost: null,
    crossValidate: null,
};
const trainResults = lstm.trainer.train(trainSet, trainOptions);
console.log(trainResults);

const testResults = [];
testResults[0] = lstm.activate([0]);
testResults[1] = lstm.activate([1]);
testResults[2] = lstm.activate([0]);
testResults[3] = lstm.activate([1]);
testResults[4] = lstm.activate([0]);
console.log(testResults);

Results in:

{ error: 0.004982436660844655, iterations: 2010, time: 384 }
[ [ 0.18288280009908592 ],
  [ 0.2948083898027347 ],
  [ 0.35061782593064206 ],
  [ 0.3900799575806566 ],
  [ 0.49454852760556606 ] ]

Which is accurate.