How to create a Image Dataset just like MNIST data

Question

I have 10000 BMP images of some handwritten digits. If i want to feed the datas to a neural network what do i need to do ? For MNIST dataset i just had to write

(X_train, y_train), (X_test, y_test) = mnist.load_data()

I am using Keras library in python . How can i create such dataset ?

Answer 1

You should write your own function to load all the images or do it like:

imagePaths = sorted(list(paths.list_images(args["testset"])))

# loop over the input images
for imagePath in imagePaths:
    # load the image, pre-process it, and store it in the data list
    image = cv2.imread(imagePath)
    image = cv2.resize(image, (IMAGE_DIMS[1], IMAGE_DIMS[0]))
    image = img_to_array(image)
    data.append(image)
    # extract the class label from the image path and update the
    # labels list


data = np.array(data, dtype="float") / 255.0

Answer 2

numpy can save array to file as binary numpy save

import numpy as np

def save_data():
  [images, labels] = read_data()
  outshape = len(images[0])
  npimages = np.empty((0, outshape), dtype=np.int32)
  nplabels = np.empty((0,), dtype=np.int32)

  for i in range(len(labels)):
      label = labels[i]
      npimages = np.append(npimages, [images[i]], axis=0)
      nplabels = np.append(nplabels, y)

  np.save('images', npimages)
  np.save('labels', nplabels)


def read_data():
  return [np.load('images.npy'), np.load('labels.npy')]

Answer 3

You can either write a function that loads all your images and stack them into a numpy array if all fits in RAM or use Keras ImageDataGenerator (https://keras.io/preprocessing/image/) which includes a function flow_from_directory. You can find an example here https://gist.github.com/fchollet/0830affa1f7f19fd47b06d4cf89ed44d.