Deep Neural Networks
Training your first artificial neural network
In the first lesson you will learn how to get started with deep learning using Keras and Tensorflow. You get to train your own deep neural network that works wih brain imaging data.
Here is the code for a deep neural network in Keras. Exciting stuff!
# Visualize training history from keras.models import Sequential from keras.layers import Dense, BatchNormalization from keras.layers import Dropout from keras.optimizers import RMSprop from keras import optimizers import matplotlib.pyplot as plt import pandas as pd import numpy as np import numpy import keras # Remove warnings import warnings warnings.filterwarnings('ignore') # fix random seed for reproducibility seed = 7 numpy.random.seed(seed) dataset = pd.read_csv("https://www.floydhub.com/viewer/data/res7UHjG5WSPgPStBT84xW/ID-OXY-20.csv") # split into input (X) and output variables # Remove missing values dataset = dataset.dropna() # make a label dataset dataset["Label"] = dataset["Mark"] # change rest values to # default mode dataset["Mark"][dataset["Mark"] == "REST"] = 0 # task positive network dataset["Mark"][dataset["Mark"] == "ADDITION"] = 1 dataset["Mark"][dataset["Mark"] == "PASSTHOUGHT"] = 2 dataset["Mark"][dataset["Mark"] == "JUNK"] = 3 # remove the JUNK data dataset = dataset[dataset.Mark != 2] dataset = dataset[dataset.Mark != 3] # shuffle the data dataset = dataset.sample(frac=1) # 52 broadmann areas data X = np.array(dataset.ix[:, :'CH52']) # default mode network or task positive network Y = np.array([[1,0] if i == 0 else [0,1] for i in dataset.Mark]) # Dropout - the number of neurons removed at each layers, who are readded when testing # Batch size - the number of data points added at each time, affects training time # Epochs - the number of training/test sessions # create model model = Sequential() # makes the values between 0 and 1 model.add(BatchNormalization(input_shape=(52,))) model.add(Dropout(0.3)) model.add(Dense(100, init="normal", activation='relu')) model.add(Dropout(0.3)) model.add(Dense(100, init="normal", activation='relu')) model.add(Dropout(0.3)) model.add(Dense(100, init="normal", activation='relu')) model.add(Dense(2, init="normal", activation='softmax')) # Compile model model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy']) # Fit the model history = model.fit(X, Y, validation_split=0.5, nb_epoch=50, batch_size=50, verbose=1) # list all data in history print(history.history.keys()) # summarize history for accuracy plt.plot(history.history['acc']) plt.plot(history.history['val_acc']) plt.title('model accuracy') plt.ylabel('accuracy') plt.xlabel('epoch') plt.legend(['train', 'test'], loc='upper left') plt.show() # summarize history for loss plt.plot(history.history['loss']) plt.plot(history.history['val_loss']) plt.title('model loss') plt.ylabel('loss') plt.xlabel('epoch') plt.legend(['train', 'test'], loc='upper left') plt.show()
What do I need to do in order to get started with machine learning?
Machine Learning Map
AI is a fast moving discipline
Machine learning and AI is a huge discipline that is always growing. However it’s easy to get started and it’s moving really fast so you can just grab on and stay on the bleeding edge of technology.
Google open sourcing Tensorflow https://www.tensorflow.org/ made it much easier to get started in machine learning. What would have taken months to do ten years ago can now be accomplished in days.
Even better, Keras came along as an abstraction on top of Tensorflow for deep learning, one of the most advanced parts of deep learning.
Types of machine learning
There are many different types of machine learning. The most advances in machine learning have been done using artificial neural networks, so if you want to start out doing just one thing, start out with artificial neural networks.
Here is a list of different types of machine learning.
Decision tree learning Association rule learning Artificial neural networks Deep learning Inductive logic programming Support vector machines Clustering Bayesian networks Reinforcement learning Representation learning Similarity and metric learning Sparse dictionary learning Genetic algorithms Rule-based machine learning Learning classifier systems
What you need to know about data and machine learning
Machine learning is a way to make advanced statistical models using math. It’s a way to make a computer guess. For machine learning to work you need good data, the more data the better. So before you go further it might be good to take a moment and think about data. The reason why machine learning models are becoming so good is that we have so much data. However machine learning models can’t perform magic (even thought it might seem like it).
Garbage in = Garbage out Take a moment and think about what problem you are trying to solve? What data do you need for solving your problem?
Classic machine learning datasets MNIST MNIST is a dataset of handwritten digits. It is the hello world of machine learning. The goal with MNIST is to classify (guess) which handwritten digit is displayed by use of a machine learning model.
IRIS IRIS is another equally famous machine learning dataset. Iris is a dataset for classifying different types of flowers.
Open Datasets from Kaggle Here are places you can find ready made datasets for machine learning http://kaggle.com/
Here is a list of all the datasets available at Kaggle with a link on where to download them.
Dataset Link IMDB
https://www.kaggle.com/deepmatrix/imdb-5000-movie-dataset Soccer scores dataset https://www.kaggle.com/hugomathien/soccer Credit card fraud detection
https://www.kaggle.com/dalpozz/creditcardfraud Human resources analytics
https://www.kaggle.com/openfoodfacts/world-food-facts Climate change
https://www.kaggle.com/berkeleyearth/climate-change-earth-surface-temperature-data Daily news for stock market predictions https://www.kaggle.com/aaron7sun/stocknews New York Stock Exchange https://www.kaggle.com/dgawlik/nyseUS Stocks Fundamentals
https://www.kaggle.com/usfundamentals/us-stocks-fundamentals Bitcoin historical data https://www.kaggle.com/mczielinski/bitcoin-historical-data Adult census income, predict earning based on census https://www.kaggle.com/uciml/adult-census-income House sales in King County https://www.kaggle.com/harlfoxem/housesalesprediction EEG brain waves https://www.kaggle.com/wanghaohan/eeg-brain-wave-for-confusion Synchronized brain wave dataset https://www.kaggle.com/berkeley-biosense/synchronized-brainwave-dataset CT medical image analysis https://www.kaggle.com/kmader/siim-medical-image-analysis-tutorial Structural MRI Dataset https://www.kaggle.com/ilknuricke/neurohackinginrimages Circadian rhythm in the brain https://www.kaggle.com/kmader/circadian-rhythm-in-the-brain Complete genome dataset https://www.kaggle.com/zusmani/mygenome Question answers dataset 100 000 + questions https://www.kaggle.com/stanfordu/stanford-question-answering-dataset What is a note https://www.kaggle.com/juliancienfuegos/what-is-a-note Health nutrition and population across the world https://www.kaggle.com/theworldbank/health-nutrition-and-population-statistics
Scrape data for machine learning Scraping is another way to get data. Scraping means to request data from websites and store it offline for later analysis.
Here you can learn more about scraping.
What is an artificial neural network? Machine Learning Map
The part of machine learning that has been making the most progress in the last years are artificial neural networks, a form of machine learning that is loosely modeled on how the neurons in our brain works. Artificual neural network has been proved really effective at making progress in many different domains.
For getting started with artificial neural networks, Keras is an excellent library built on top of the Tensorflow API. https://keras.io/
Here is a good video explaining artificial neural networks https://www.youtube.com/watch?v=ILsA4nyG7I0
The math of machine learning Image of Activation function The truth is that math is a big part of machine learning but you don’t need to know any math to get started with machine learning. It’s an excellent idea to pick up the math along the way and to expand the field of machine learning you definitely need to learn math.
If you are curious, here are some resources for looking at the math behind deep learning.
The different math sources AI draws math from three different sources, linear algebra, calculus and statistics.
Neural network from scratch Building a simple neural network from scratch. Walkthrough and experiments. http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/
Lets train our first neural network! To train a neural network we need two things.
A dataset with a classifier to train on A mathematical neural network Task default dataset from my brain To make it fun, lets use a dataset that takes input data from my brain measured with laser sensors to try to determine what brain region is active, the default mode network or the task positive network.
The data takes measurement from two probes placed on my forehead and returns eight values of oxygenated and deoxygenated blood.