Computer vision

Image Classification

Image classification is the ability to make computer models that use vision.

how to improve model predictions

To acquire a few hundreds or thousands of training images belonging to the classes you are interested in, one possibility would be to use the Flickr API to download pictures matching a given tag, under a friendly license. In our examples we will use two sets of pictures, which we got from Kaggle: 1000 cats and 1000 dogs (although the original dataset had 12,500 cats and 12,500 dogs, we just took the first 1000 images for each class). We also use 400 additional samples from each class as validation data, to evaluate our models. That is very few examples to learn from, for a classification problem that is far from simple. So this is a challenging machine learning problem, but it is also a realistic one: in a lot of real-world use cases, even small-scale data collection can be extremely expensive or sometimes near-impossible (e.g. in medical imaging). Being able to make the most out of very little data is a key skill of a competent data scientist. The data can be downloaded from kaggle. Because we are awesome we also stored the data on our floydhob serverhttps://www.floydhub.com/viewer/data/xLHRt9d9UnipGEEr4FkWdF/ https://www.kaggle.com/c/dogs-vs-cats Step 1. Download the data from kaggle and sort the data into two different folders Train containing 1000 cat images and 1000 dog images and Validation containing 400 images Here is how you should structure your folders.

Preprocessing

The greatest part of machine learning is fitting your data into the right format. This code helps put your data in the proper folders / train and / validation and at the percentage you want and changing the input dimensions of the images.

preprocessing

import os import errno import itertools import scipy.misc import numpy as np

curr_dir = os.getcwd() proj_dir = os.path.normpath(os.path.join(curr_dir)) image_dir = ‘data/train’ input_filepath = os.path.normpath(os.path.join(proj_dir, image_dir))

validation_dir = ‘data/processed/validation’ train_dir = ‘data/processed/train’

class_dict = {‘dog’: 0, ‘cat’: 1}

def put_classes_into_separate_folders(parent_dir, images): make_sure_path_exists(parent_dir)

def make_sure_path_exists(path): try: os.makedirs(path) except OSError as exception: if exception.errno != errno.EEXIST: raise

train_shuffled_images = [] test_shuffled_images = [] def create_train_validation_shuffle_images(): test_percentage = 0.2 for key, _ in class_dict.items(): class_images = [x for x in os.listdir(input_filepath) if x.startswith(key + “.”)] k = int(len(class_images) * test_percentage) test_shuffled_images = class_images[0:k] train_shuffled_images = class_images[k:] return train_shuffled_images, test_shuffled_images

train_images = [] validation_images = [] def resize_of_images(): train_images = [scipy.misc.imresize(scipy.misc.imread(input_filepath + ‘/’ + image), (100,50)) for image in train_shuffled_images] validation_images = [scipy.misc.imresize(scipy.misc.imread(input_filepath + ‘/’ + image), (100,50)) for image in test_shuffled_images] return train_images, validation_images

def put_class_images_in_folders(save_dir, image_files, class_feature): counter = 0 class_counter = 0 first = True

make_sure_path_exists(save_dir)

for image in image_files:
    counter += 1

    if (counter % int(len(image_files) / len(class_feature)) == 0) and (first == False) != (counter == int(len(image_files))):
        class_counter += 1
    first = False

    class_dir = os.path.join(save_dir,class_feature[class_counter])
    make_sure_path_exists(class_dir)

    save_image = os.path.join(class_dir,'{}_{}.jpeg'.format(class_feature[class_counter],"".join((map(str,np.random.randint(0,9,8))))))

    scipy.misc.imsave(save_image, image)

if name == ‘main‘: train_shuffled_images, test_shuffled_images = create_train_validation_shuffle_images() train_images, test_images = resize_of_images() keys = [] for k, _ in class_dict.items(): keys.append(k) print(input_filepath) put_class_images_in_folders(input_filepath + ‘/train/’, train_images, keys) put_class_images_in_folders(input_filepath + ‘/validation/’, test_images, keys)

Convolutional neural networks

Convolutional neural networks are neural network that are specialised in image classification. Their functioning is inspired by the mammalian visual cortex. ”’This script goes along the blog post “Building powerful image classification models using very little data” from blog.keras.io. It uses data that can be downloaded at: https://www.kaggle.com/c/dogs-vs-cats/data In our setup, we:

  • created a data/ folder
  • created train/ and validation/ subfolders inside data/
  • created cats/ and dogs/ subfolders inside train/ and validation/
  • put the cat pictures index 0-999 in data/train/cats
  • put the cat pictures index 1000-1400 in data/validation/cats
  • put the dogs pictures index 12500-13499 in data/train/dogs
  • put the dog pictures index 13500-13900 in data/validation/dogs So that we have 1000 training examples for each class, and 400 validation examples for each class. In summary, this is our directory structure:
data/
    train/
        dogs/
            dog001.jpg
            dog002.jpg
            ...
        cats/
            cat001.jpg
            cat002.jpg
            ...
    validation/
        dogs/
            dog001.jpg
            dog002.jpg
            ...
        cats/
            cat001.jpg
            cat002.jpg
            ...

”’

from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K


# dimensions of our images.
img_width, img_height = 150, 150

# flowdhub folder
flowdhub = 'https://www.floydhub.com/viewer/data/xLHRt9d9UnipGEEr4FkWdF/'
flowhubtrain = 'https://www.floydhub.com/viewer/data/xLHRt9d9UnipGEEr4FkWdF/train'
flowdhubvalidation = 'https://www.floydhub.com/viewer/data/xLHRt9d9UnipGEEr4FkWdF/validation'

train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 2
batch_size = 16

if K.image_data_format() == 'channels_first':
    input_shape = (3, img_width, img_height)
else:
    input_shape = (img_width, img_height, 3)

model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size)

model.save_weights('first_try.h5')
saved_model = model.to_json()
with open('first_try.json','w') as f:
    f.write(saved_model)

Found 2002 images belonging to 2 classes. Found 801 images belonging to 2 classes. Epoch 1/2 125/125 [==============================] – 71s – loss: 0.6986 – acc: 0.5530 – val_loss: 0.6580 – val_acc: 0.6025 Epoch 2/2 125/125 [==============================] – 78s – loss: 0.6714 – acc: 0.5980 – val_loss: 0.6194 – val_acc: 0.6701

Saving a trained model

# Saving the model weights
model.save_weights('first_try.h5')
saved_model = model.to_json()
with open('first_try.json','w') as f:
    f.write(saved_model)


# Loading a saved model
```python
# Loading the saved model
from keras.models import model_from_json
# Load trained model
# load json and create model
json_file = open('first_try.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
global loaded_model
loaded_model = model_from_json(loaded_model_json)
# load weights into new model
loaded_model.load_weights("first_try.h5")
print("Loaded model from disk")

Reshaping our trained image

import scipy.misc

testimage = scipy.misc.imresize(scipy.misc.imread('data/validation/cats/cat.1211.jpg'), (150, 150))
testimage = testimage.reshape((1,) + testimage.shape)
# checking the shape of our new image
testimage.shape
(1, 150, 150, 3)

Lets test our prediction on a test image

prediction = loaded_model.predict(testimage) print(prediction)

if (prediction[0] == 1): print(“Cat”) one = True else: print(“Dog”)

[[ 1.]] Cat Our classification works but it isn’t that good. What can we do to improve our predictions? Hint. You could probably change settings for how you train your model.

'''This script goes along the blog post
"Building powerful image classification models using very little data"
from blog.keras.io.
It uses data that can be downloaded at:
https://www.kaggle.com/c/dogs-vs-cats/data
In our setup, we:
- created a data/ folder
- created train/ and validation/ subfolders inside data/
- created cats/ and dogs/ subfolders inside train/ and validation/
- put the cat pictures index 0-999 in data/train/cats
- put the cat pictures index 1000-1400 in data/validation/cats
- put the dogs pictures index 12500-13499 in data/train/dogs
- put the dog pictures index 13500-13900 in data/validation/dogs
So that we have 1000 training examples for each class, and 400 validation examples for each class.
In summary, this is our directory structure:

data/
    train/
        dogs/
            dog001.jpg
            dog002.jpg
            ...
        cats/
            cat001.jpg
            cat002.jpg
            ...
    validation/
        dogs/
            dog001.jpg
            dog002.jpg
            ...
        cats/
            cat001.jpg
            cat002.jpg
            ...

'''

from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K


# dimensions of our images.
img_width, img_height = 150, 150

# flowdhub folder
flowdhub = 'https://www.floydhub.com/viewer/data/xLHRt9d9UnipGEEr4FkWdF/'
flowhubtrain = 'https://www.floydhub.com/viewer/data/xLHRt9d9UnipGEEr4FkWdF/train'
flowdhubvalidation = 'https://www.floydhub.com/viewer/data/xLHRt9d9UnipGEEr4FkWdF/validation'

train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 2400
nb_validation_samples = 400
epochs = 10
batch_size = 16

if K.image_data_format() == 'channels_first':
    input_shape = (3, img_width, img_height)
else:
    input_shape = (img_width, img_height, 3)

model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size)

model.save_weights('first_try.h5')
saved_model = model.to_json()
with open('first_try.json','w') as f:
    f.write(saved_model)