Introduction to Deep Learning with TensorFlow

Welcome to part two of Deep Learning with Neural Networks and TensorFlow, and part 44 of the Machine Learning tutorial series. In this tutorial, we are going to be covering some basics on what TensorFlow is, and how to begin using it.

Libraries like TensorFlow and Theano are not simply deep learning libraries, they are libraries *for* deep learning. They are actually just number-crunching libraries, much like Numpy is. The difference is, however, a package like TensorFlow allows us to perform specific machine learning number-crunching operations like derivatives on huge matricies with large efficiency. We can also easily distribute this processing across our CPU cores, GPU cores, or even multiple devices like multiple GPUs. But that's not all! We can even distribute computations across a distributed network of computers with TensorFlow. So, while TensorFlow is mainly being used with machine learning right now, it actually stands to have uses in other fields, since really it is just a massive array manipulation library.

What is a tensor? Up to this point in the machine learning series, we've been working mainly with vectors (numpy arrays), and a tensor can be a vector. Most simply, a tensor is an array-like object, and, as you've seen, an array can hold your matrix, your vector, and really even a scalar.

At this point, we just simply need to translate our machine learning problems into functions on tensors, which is possible with just about every single ML algorithm. Consider the neural network. What does a neural network break down into?

We have data (X), weights (w), and thresholds (t). Are all of these tensors? X will be the dataset (an array), so that's a tensor. The weights are also an array of weight values, so they're tensors too. Thresholds? Same as weights. Thus, our neural network is indeed a function of X,w, and t, or f(Xwt), so we are all set and can certainly use TensorFlow, but how?

TensorFlow works by first defining and describing our model in abstract, and then, when we are ready, we make it a reality in the session. The description of the model is what is known as your "Computation Graph" in TensorFlow terms. Let's play with a simple example. First, let's construct the graph:

import tensorflow as tf

# creates nodes in a graph
# "construction phase"
x1 = tf.constant(5)
x2 = tf.constant(6)

So we have some values. Now, we can do things with those values, such as multiplication:

result = tf.mul(x1,x2)

Notice that the output is just an abstract tensor still. No actual calculations have been run, only operations created. Each operation, or "op," in our computation graph is a "node" in the graph.

To actually see the result, we need to run the session. Generally, you build the graph first, then you "launch" the graph:

# defines our session and launches graph
sess = tf.Session()
# runs result

We can also assign the output from the session to a variable:

output =

When you are finished with a session, you need to close it in order to free up the resources that were used:


After closing, you can still reference that output variable, but you cannot do something like:

...which would just return an error. Another option you have is to utilize Python's with statement:

with tf.Session() as sess:
    output =

If you are not familiar with what this does, basically, it will use the session for the block of code following the statement, and then automatically close the session when done, the same way it works if you open a file with the with statement.

You can also use TensorFlow on multiple devices, and even multiple distributed machines. An example for running some computations on a specific GPU would be something like:

with tf.Session() as sess:
  with tf.device("/gpu:1"):
    matrix1 = tf.constant([[3., 3.]])
    matrix2 = tf.constant([[2.],[2.]])
    product = tf.matmul(matrix1, matrix2)

Code from: TensorFlow docs. The tf.matmul is a matrix multiplication function.

The above code would run the calcuation on the 2nd system GPU. If you installed the CPU version with me, then this isn't currently an option, but you should still be aware of the possibility down the line. The GPU version of TensorFlow requires CUDA to be properly set up (along with needing a CUDA-enabled GPU). I have a few CUDA enabled GPUs, and would like to eventually cover their use as well, but that's for another day!

Now that we have the basics of TensorFlow down, I invite you down the rabbit hole of creating a Deep Neural Network in the next tutorial. If you need to install TensorFlow, the installation process is very simple if you are on Mac or Linux. On Windows, not so much. The next tutorial is optional, and it is just us installing TensorFlow on a Windows machine.

The next tutorial:

  • Practical Machine Learning Tutorial with Python Introduction
  • Regression - Intro and Data
  • Regression - Features and Labels
  • Regression - Training and Testing
  • Regression - Forecasting and Predicting
  • Pickling and Scaling
  • Regression - Theory and how it works
  • Regression - How to program the Best Fit Slope
  • Regression - How to program the Best Fit Line
  • Regression - R Squared and Coefficient of Determination Theory
  • Regression - How to Program R Squared
  • Creating Sample Data for Testing
  • Classification Intro with K Nearest Neighbors
  • Applying K Nearest Neighbors to Data
  • Euclidean Distance theory
  • Creating a K Nearest Neighbors Classifer from scratch
  • Creating a K Nearest Neighbors Classifer from scratch part 2
  • Testing our K Nearest Neighbors classifier
  • Final thoughts on K Nearest Neighbors
  • Support Vector Machine introduction
  • Vector Basics
  • Support Vector Assertions
  • Support Vector Machine Fundamentals
  • Constraint Optimization with Support Vector Machine
  • Beginning SVM from Scratch in Python
  • Support Vector Machine Optimization in Python
  • Support Vector Machine Optimization in Python part 2
  • Visualization and Predicting with our Custom SVM
  • Kernels Introduction
  • Why Kernels
  • Soft Margin Support Vector Machine
  • Kernels, Soft Margin SVM, and Quadratic Programming with Python and CVXOPT
  • Support Vector Machine Parameters
  • Machine Learning - Clustering Introduction
  • Handling Non-Numerical Data for Machine Learning
  • K-Means with Titanic Dataset
  • K-Means from Scratch in Python
  • Finishing K-Means from Scratch in Python
  • Hierarchical Clustering with Mean Shift Introduction
  • Mean Shift applied to Titanic Dataset
  • Mean Shift algorithm from scratch in Python
  • Dynamically Weighted Bandwidth for Mean Shift
  • Introduction to Neural Networks
  • Installing TensorFlow for Deep Learning - OPTIONAL
  • Introduction to Deep Learning with TensorFlow
  • Deep Learning with TensorFlow - Creating the Neural Network Model
  • Deep Learning with TensorFlow - How the Network will run
  • Deep Learning with our own Data
  • Simple Preprocessing Language Data for Deep Learning
  • Training and Testing on our Data for Deep Learning
  • 10K samples compared to 1.6 million samples with Deep Learning
  • How to use CUDA and the GPU Version of Tensorflow for Deep Learning
  • Recurrent Neural Network (RNN) basics and the Long Short Term Memory (LSTM) cell
  • RNN w/ LSTM cell example in TensorFlow and Python
  • Convolutional Neural Network (CNN) basics
  • Convolutional Neural Network CNN with TensorFlow tutorial
  • TFLearn - High Level Abstraction Layer for TensorFlow Tutorial
  • Using a 3D Convolutional Neural Network on medical imaging data (CT Scans) for Kaggle
  • Classifying Cats vs Dogs with a Convolutional Neural Network on Kaggle
  • Using a neural network to solve OpenAI's CartPole balancing environment