
Introduction
The goal of this article is to cover the following topics:
- Introduction to Tensors
- Graphs, variables & operations
- Problem resolution with TensorFlow
What is TensorFlow
TensorFlow is a framework developed and maintained by Google that enables mathematical operations to be performed in an optimized way on a CPU or GPU. We are going to focus on the GPU since it is the fastest way we have to train a deep neural network.
Why Tensorflow?
- Because of its flexibility and scalability
- Because of its popularity

The key features that make TensorFlow the most popular Deep Learning library are:
- TensorFlow uses tensors to perform the operations.
- In TensorFlow, you first define the activities to be performed (build the graph), and then execute them (execute the graph). This allows the process to be optimized to the task at hand, reducing greatly the computation time.
- TensorFlow enables code to be run in parallel or on one or more GPUs.
Okay, but what’s a tensor?
Although tensors were invented by physicists to be able to describe interactions, in the field of Artificial Intelligence they can be understood simply as containers of numbers.

Let’s put all this now into practice. We will use code some tensors in python in order to understand better what they are and how they work.
Tensors In Practice
Let’s imagine that we want to store the average grade of a student. We will use a 0D tensor, which is just a simple number or scalar.
import numpy as np
tensor_0D = np.array(5)
print("Average grade: n{}".format(tensor_0D))
print("Tensor dimensions: n{}".format(tensor_0D.ndim))

Let’s try now to store the grade of every subject that this student courses. We can use a 1D tensor to do so:
tensor_1D = np.array([4,6,8])
print("Subject grades: n{}".format(tensor_1D))
print("Tensor dimensions: n{}".format(tensor_0D.ndim))

But wait… What if we want to store each grade of every exam the student took on each subject? How could we do this if each subject had 3 exams?
By using a 2D tensor! Which, as we have seen before, is a matrix.
# 2D Tensor (matrix)
tensor_2D = np.array([[0, 1, 1], # Subject 1
[2, 3, 3], # Subject 2
[1, 3, 2]]) # Subject 3
print("Exam grades are:n{}".format(tensor_2D))
print("Subject 1:n{}".format(tensor_2D[0]))
print("Subject 2:n{}".format(tensor_2D[1]))
print("Subject 3:n{}".format(tensor_2D[2]))
print("Tensor dimensions: n{}".format(tensor_2D.ndim))

We want now to store the grades of the subjects (which are annual) for four quarters, so it will be easier to access them if necessary in a future, don’t you think? How do you think we could organize them?
What if we add a dimension to our 2D tensor that indicates the quarter?
We would get a 3D tensor (3D matrix or cube).
tensor_3D = np.array([[[0, 1, 1], # First quarter
[2, 3, 3],
[1, 3, 2]],
[[1, 3, 2], # Second quarter
[2, 4, 2],
[0, 1, 1]]])
print("Exam grades per quarter are:n{}".format(tensor_3D))
print("First quarter:n{}".format(tensor_3D[0]))
print("Second quarter:n{}".format(tensor_3D[1]))
print("Tensor dimensions: n{}".format(tensor_3D.ndim))

What if we add a dimension to our tensor, so that we can have the grades per term of each subject for each student?
It will be a 4D tensor (3D matrix vector or cube vector).
tensor_4D = np.array([[[[0, 1, 1], # Jacob
[2, 3, 3],
[1, 3, 2]],
[[1, 3, 2],
[2, 4, 2],
[0, 1, 1]]],
[[[0, 3, 1], # Christian
[2, 4, 1],
[1, 3, 2]],
[[1, 1, 1],
[2, 3, 4],
[1, 3, 2]]],
[[[2, 2, 4], # Sofia
[2, 1, 3],
[0, 4, 2]],
[[2, 4, 1],
[2, 3, 0],
[1, 3, 3]]]])
print("The grades of each student are:n{}".format(tensor_4D))
print("Jacob's grades:n{}".format(tensor_4D[0]))
print("Christian's grades:n{}".format(tensor_4D[1]))
print("Sofia's grades:n{}".format(tensor_4D[2]))
print("Tensor dimensions: n{}".format(tensor_4D.ndim))

And so we could go on to infinity adding dimensions to our tensors, to be able to store more data.
To give you an idea of how often tensors are used in the world of Deep Learning, the most common types of tensors are:
- 3D tensors: used in time series.
- 4D-Tensors: used with images.
- 5D tensioners: used with videos.
Normally, one of the dimensions will be used to store the samples of each type of data. For example, with images:
If we want to store 64 RGB images of 224×224 pixels, we will need a 3D matrix vector, or what is the same, a 4D tensor. How many dimensions do we need?
We have 64 images of 224 pixels x 224 pixels x 3 channels (R, G and B).
Therefore: (64, 224, 224, 3)
If you want to go deeper into tensors or more examples, here is a very good resource for it: Tensors illustrated with cats
We said before that in TensorFlow, first you define the operations to be carried out and then you execute them. To do this, you use a graph.
And what is a graph?
A simple example of a sum of a + b.

And here is a more complex example, for now, you do not have to understand completely, but this is just a snippet of how intricate they can get.

TensorFlow into Practice
First of all, we need to define and understand some basic concepts of TensorFlow (from now on TF):
- tf.Graph: represent a set of tf.Operations
- tf.Operation: Are the operations determined by the equations defined by us
- tf.Tensor: Where we store the results of tf.Operations
In the beginning, tf.Graph is transparent to us, because there is one default graph where are added all our defined operations. It is called tf.get_default_graph().
# We import the Tensorflow package and matplotlib for the charts
import tensorflow as tf
import matplotlib.pyplot as plt
%matplotlib inline
Let’s start with something very simple, just a simple multiplication in Tensorflow.
# Variables definition
x = tf.constant(6)
y = tf.constant(8)
# Operations definition
result = tf.multiply(x, y)
print(result)

As you can see, it didn’t give us back the result. What it has done so far is create the network.
To give you an example, it’s like riding in a car. Now we have it assembled, but it still doesn’t do what it was designed to do, move. To do that, we should turn it on. To do so, we will use tf.Session().
sess = tf.Session()
output = sess.run(result)
print(output)

To be able to visualize the graph, we should define a couple of functions.
from IPython.display import clear_output, Image, display, HTML
def strip_consts(graph_def, max_const_size=32):
"""Strip large constant values from graph_def."""
strip_def = tf.GraphDef()
for n0 in graph_def.node:
n = strip_def.node.add()
n.MergeFrom(n0)
if n.op == 'Const':
tensor = n.attr['value'].tensor
size = len(tensor.tensor_content)
if size > max_const_size:
tensor.tensor_content = "<stripped %d bytes>"%size
return strip_def
def show_graph(graph_def, max_const_size=32):
"""Visualize TensorFlow graph."""
if hasattr(graph_def, 'as_graph_def'):
graph_def = graph_def.as_graph_def()
strip_def = strip_consts(graph_def, max_const_size=max_const_size)
code = """
<script>
function load() {{
document.getElementById("{id}").pbtxt = {data};
}}
</script>
<link rel="import" href="https://tensorboard.appspot.com/tf-graph-basic.build.html" onload=load()>
<div style="height:600px">
<tf-graph-basic id="{id}"></tf-graph-basic>
</div>
""".format(data=repr(str(strip_def)), id='graph'+str(np.random.rand()))
iframe = """
<iframe seamless style="width:1200px;height:620px;border:0" srcdoc="{}"></iframe>
""".format(code.replace('"', '&quot;'))
display(HTML(iframe)
And now we will show the previously defined graph:

As we can see, the graph consist of 2 nodes of constant type and one of operator type (multiplication). However, their names are not quite indicatives. Let’s change this:
x2 = tf.constant(5.0, name='x2')
y2 = tf.constant(6.0, name='y2')
result = tf.multiply(x2, y2)
# Let's see what it looks like now:
show_graph(tf.get_default_graph().as_graph_def())

Once we have performed the operation, we should close the session to free the resources with sess.close().
Lastly, we can also indicate to TF the GPU that we want it to execute the operations. To do so, we can print a list of the available devices:
from tensorflow.python.client import device_lib
def get_available_devices():
local_device_protos = device_lib.list_local_devices()
return [x.name for x in local_device_protos]
print(get_available_devices())

We will select the GPU:0 and perform the multiplication [3 3] by [2 2], which may result in 3×3 +2×2 = 12. Let’s check it out:
By using the with as : we make python to free the resources of the TF session by itself:
with tf.Session() as sess:
with tf.device("/GPU:0"):
matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
product = tf.matmul(matrix1, matrix2)
output = sess.run(product)
print(output)

Let’s create now a 1D Tensor of 32 values equally sparsed between -5 and 5:
n_values = 64
x = tf.linspace(-5.0, 5.0, n_values)
sess = tf.Session()
result = sess.run(x)
print(result)

In addition to sess.run(_), there are other ways to evaluate Tensors
x.eval(session=sess)

We need to always remember to close the session:
sess.close()
We can also use an interactive session, which can help us by not having to be constantly calling the .run() to have the results executed:
sess = tf.InteractiveSession()
x.eval()

Let’s see now a tf.Operation, to do so, we will use "x" to create and visualize a gaussian distribution. Its formula is:

sigma = 1.0
mean = 0
# To implement the gaussian distributio formula:
g1d = (tf.exp(tf.negative(tf.pow(x - mean, 2.0) / (2.0 * tf.pow(sigma, 2.0)))) *
(1.0 / (sigma * tf.sqrt(2.0 * 3.1415))))
# to check that this operation has been succesuflly included in our tf.Graph
if g1d.graph is tf.get_default_graph():
print('All good')

plt.plot(g1d.eval())

# to see the dimensions
print(g1d.get_shape())
print(type(g1d.get_shape()))
print(g1d.get_shape().as_list())
print(type(g1d.get_shape().as_list()))

There will be times when we don’t know the dimensions of a variable until the operation that returns its value is executed. For these cases, we can use tf.shape(variable), which returns a Tensor that will calculate at run time the dimensions of our result.
This is known as ‘static shape’ and ‘dynamic shape’, where static is calculated taking into account the dimensions of the Tensors and Operations involved, and the dynamics at execution time.
What happens if we define x as a ‘placeholder’? A placeholder is like a reserve, it indicates that there will be a tensor there, but it is not necessary to define it at that moment. For example, by defining:
x = tf.placeholder(tf.int32, shape=[5])
We know that x will hold a 1D 5 dimensional tensor, as confirmed by x.get_shape():
print(x.get_shape())

But we don’t know what values will form it until we tell it.
Differences between placeholder and variable:
Variables
They are used to accommodate parameters that are learned during training.
Therefore, the values are derived from training – They require an initial value to be assigned (can be random)
Placeholders
Reserve space for data (e.g. for the pixels in an image)
They do not require a value to be assigned to start (although they can)
The obtained value, 5, it is the static value of the x dimensions. But, what happens if we apply a tf.unique() to x?
y, _ = tf.unique(x)
print(y.get_shape())

What happens is that tf.unique() returns the unique values of x, which are not known at first, since x is defined as a placeholder, and a placeholder does not have to be defined until the moment of execution, as we said before. In fact, let’s see what happens if we feed "x" with two different values:
with tf.Session() as sess:
print(sess.run(y, feed_dict={x: [0, 1, 2, 3, 4]}).shape)
print(sess.run(y, feed_dict={x: [0, 0, 0, 0, 1]}).shape)

Look at that! The size of y changes depending on what tf.unique() returns. This is called "dynamic shape", and it is always defined, it will never return a question by answer. Because of this, TensorFlow supports operations like tf.unique() which can have variable size results.
So, now you know, every time that you use operations with output variables, you will need to use tf.shape(variable) to calculate the dynamic shape of a tensor.
sy = tf.shape(y)
# Returns a list with the dimensions
print(sy.eval(feed_dict={x: [0, 1, 2, 3, 4]}))
print(sy.eval(feed_dict={x: [0, 0, 0, 0, 1]}))
# We access the dimension of interest
print(sy.eval(feed_dict={x: [0, 1, 2, 3, 4]})[0])
print(sy.eval(feed_dict={x: [0, 0, 0, 0, 1]})[0])
Now we can perform the operations after taking into account the size of the output of the operation, which we do not know in the first instance.
print(tf.shape(y).eval(feed_dict={x: [0, 1, 4, 1, 0]}))
print(type(tf.shape(y).eval(feed_dict={x: [0, 1, 4, 1, 0]})))
print(tf.stack([y, y[::-1], tf.range(tf.shape(y)[0])]).eval(feed_dict={x: [0, 1, 4, 1, 0]}))

Let’s see now a Gaussian distribution in 2D
g1d_r = tf.reshape(g1d, [n_values, 1])
print(g1d.get_shape().as_list())
print(g1d_r.get_shape().as_list())
# We multiply the row vector of the 1D Gaussian by the column to obtain the 2D version
g2d = tf.matmul(tf.reshape(g1d, [n_values, 1]), tf.reshape(g1d, [1, n_values]))
# To visualize it
plt.imshow(g2d.eval())

To see the list of the operations included in our tf.Graph
ops = tf.get_default_graph().get_operations()
print([op.name for op in ops])

Conclusion
As always, I hope you ** enjoyed the post, and that you have learned the basics of Tensors and TensorFlow and how they are used**.
If you liked this post then you can take a look at my other posts on Data Science and Machine Learning here .
If you want to learn more about Machine Learning and Artificial Intelligence follow me on Medium, and stay tuned for my next posts!