Convolutional neural network Conv1d input shape

Question

I am trying to create a CNN to classify data. My Data is X[N_data, N_features] I want to create a neural net capable of classifying it. My problem is concerning the input shape of a Conv1D for the keras back end.

I want to repeat a filter over.. let say 10 features and then keep the same weights for the next ten features. For each data my convolutional layer would create N_features/10 New neurones. How can i do so? What should I put in input_shape?

def cnn_model():
model = Sequential()                                               
model.add(Conv1D(filters=1, kernel_size=10 ,strides=10,     
                  input_shape=(1, 1,N_features),kernel_initializer= 'uniform',      
                  activation= 'relu')) 
model.flatten()
model.add(Dense(N_features/10, init= 'uniform' , activation= 'relu' ))

Any advice? thank you!

Answer 1

To input a usual feature table data of shape (nrows, ncols) to Conv1d of Keras, following 2 steps are needed:

xtrain.reshape(nrows, ncols, 1)
# For conv1d statement: 
input_shape = (ncols, 1)

For example, taking first 4 features of iris dataset:

To see usual format and its shape:

iris_array = np.array(irisdf.iloc[:,:4].values)
print(iris_array[:5])
print(iris_array.shape)

The output shows usual format and its shape:

[[5.1 3.5 1.4 0.2]
 [4.9 3.  1.4 0.2]
 [4.7 3.2 1.3 0.2]
 [4.6 3.1 1.5 0.2]
 [5.  3.6 1.4 0.2]]

(150, 4)

Following code alters the format:

nrows, ncols = iris_array.shape
iris_array = iris_array.reshape(nrows, ncols, 1)
print(iris_array[:5])
print(iris_array.shape)

Output of above code data format and its shape:

[[[5.1]
  [3.5]
  [1.4]
  [0.2]]

 [[4.9]
  [3. ]
  [1.4]
  [0.2]]

 [[4.7]
  [3.2]
  [1.3]
  [0.2]]

 [[4.6]
  [3.1]
  [1.5]
  [0.2]]

 [[5. ]
  [3.6]
  [1.4]
  [0.2]]]

(150, 4, 1)

This works well for Conv1d of Keras. For input_shape (4,1) is needed.

Answer 2

@Marcin's answer might work, but might suggestion given the documentation here:

When using this layer as the first layer in a model, provide an input_shape argument (tuple of integers or None, e.g. (10, 128) for sequences of 10 vectors of 128-dimensional vectors, or (None, 128) for variable-length sequences of 128-dimensional vectors.

would be:

model = Sequential()
model.add(Conv1D(filters=1, kernel_size=10 ,strides=10,     
                  input_shape=(None, N_features),kernel_initializer= 'uniform',      
                  activation= 'relu')) 

Note that since input data (N_Data, N_features), we set the number of examples as unspecified (None). The strides argument controls the size of of the timesteps in this case.

Answer 3

Try:

def cnn_model():
    model = Sequential()                                               
    model.add(Conv1D(filters=1, kernel_size=10 ,strides=10,     
              input_shape=(N_features, 1),kernel_initializer= 'uniform',      
              activation= 'relu')) 
model.flatten()
model.add(Dense(N_features/10, init= 'uniform' , activation= 'relu' ))
....

And reshape your x to shape (nb_of_examples, nb_of_features, 1).

EDIT:

Conv1D was designed for a sequence analysis - to have convolutional filters which would be the same no matter in which part of sequence we are. The second dimension is so called features dimension where you could have a vector of multiple features at each of timesteps. One may think about sequence dimension the same as spatial dimensions and feature dimension the same as channel dimension or color dimension in Conv2D. As @putonspectacles mentioned in his comment - you may set sequence dimension to None in order to make your network input length invariant.