c++-default_random_engine creates all the time sam

Question

I am working with neural networks, and I want to create the weights randomly. SO if I create 30 Neural networks every single one of them ends up having the same weights(supposed to be random) so when i give them all the same input the output is the same, when it shouldn't. Any help?

Here is the main function

int main(){
std::vector<Improved_NN> v;
std::random_device rd;
std::default_random_engine generator(rd());
std::uniform_real_distribution<double> distribution(-1.0,1.0);
for(int i = 0; i < 30; i++)
{
    Improved_NN temp;
    temp.initialize_weights(generator, distribution);
    v.push_back(temp);
}

Board temp;

for(int i = 0; i < 30; i++)
{
    std::cout <<"\n" << v[i].executeFromExternal(temp);
}

And the initialize_weights is here:

 void Improved_NN::initialize_weights(std::default_random_engine gen,std::uniform_real_distribution<double> dist){
int k,v = 0;
for(k = 0;k<NUM_HIDDEN_1;k++){
    for(v = 0 ; v < NUM_INPUTS; v++){
        mlp_t.w_h1_i[k][v]=dist(gen);
        //std::cout<<mlp_t.w_h1_i[k][v]<<std::endl;
    }
}
for(k = 0;k<NUM_HIDDEN_2;k++){
    for(v = 0 ; v < NUM_HIDDEN_1; v++){
        mlp_t.w_h2_h1[k][v]=dist(gen);
        //std::cout<<mlp_t.w_h2_h1[k][v]<<std::endl;
    }
}
for(k = 0;k<NUM_HIDDEN_3;k++){
    for(v = 0 ; v < NUM_HIDDEN_2; v++){
        mlp_t.w_h3_h2[k][v]=dist(gen);
        //std::cout<<mlp_t.w_h3_h2[k][v]<<std::endl;
    }
}
    for (int a = 0 ; a < NUM_HIDDEN_3;a++){
        mlp_t.w_o_h[0][a] = dist(gen);
        //std::cout<<mlp_t.w_o_h[0][a]<<std::endl;
    }
}

This is the output i get every time i execute.

0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458 0.521458

Thank you very much.

Answer 1

In the documentation of <random>, it is stated that one has to bind the generator to the distribution in order to reuse it.

Here is a small example:

#include <functional>
#include <iostream>
#include <random>

int main(int argc, char *argv[])
{
    std::default_random_engine rd;
    std::uniform_real_distribution<double> distribution(-1.0, 1.0);
    auto weight = std::bind(distribution, rd);
    for (int i = 0; i < 30; ++i)
    {
        std::cout << weight() << " ";
    }
    return 0;
}

Probably you can adapt it to your case, by passing weight (by reference, as @NemanjaBoric has noted) to the initialize_weights method.

Here is how to compile and run:

c++ main.cpp -o rands -std=c++11 && ./rands
-0.736924 -0.0826997 -0.562082 0.357729 0.869386 0.0388327 -0.930856 0.0594004 -0.984604 -0.866316 0.373545 0.860873 0.0538576 0.307838 0.402381 0.524396 -0.905071 -0.343532 0.512821 -0.269323 0.965101 0.506712 -0.854628 0.769414 -0.127177 -0.0445365 -0.450186 -0.666986 0.795313 -0.878871

Answer 2

Try:

 std::random_device mch;
 std::default_random_engine generator(mch());
 std::normal_distribution<double> distribution(0.0, 1.0);

Answer 3

Well, as with plain old rand() you need to provide initial seed for the generator which needs to be different value in order for generator to generate different sequence:

    std::random_device rd;
    std::default_random_engine generator(rd());

Also, as user3018144 pointed out, use single generator, instead of 30 different:

int main(){
    std::vector<Improved_NN> v;
    std::random_device rd;
    std::default_random_engine generator(rd());
    std::uniform_real_distribution<double> distribution(-1.0,1.0);

    for(int i = 0; i < 30; i++)
    {
        Improved_NN temp;

        /*Problem is here*/
        temp.initialize_weights(generator, distribution);
        v.push_back(temp);
    }

    Board temp;

    for(int i = 0; i < 30; i++)
    {
        std::cout <<"\n" << v[i].executeFromExternal(temp);
    }


//for the number of generations, do this....
}

Also, right here:

 void Improved_NN::initialize_weights(std::default_random_engine gen,std::uniform_real_distribution<double> dist){

you're passing the generators by value, which will create a copy of the existing generator. Pass by reference instead:

 void Improved_NN::initialize_weights(std::default_random_engine& gen, std::uniform_real_distribution<double>& dist){