TensorFlow on Mobile: Tutorial

On Android and iOS

TensorFlow is usually used for training huge models from tons of data but no one can ignore the emerging market of smartphones and the need to make our future “Artificially Intelligent”. And people who can’t wait for the future and love Machine Learning are pushing the boundaries by making tools, which target the multi-trillion dollar market.

Source

You are a part of this multi-trillion dollar market because you might be reading this on your Android or iOS right now or you have one in your pocket, whichever might be the case, you know how future oriented Machine Learning on Mobile is…

I’ll be writing 2 different tutorials on how to run your Machine Learning Models on your Android and iOS devices.

This is the First Tutorial.

This tutorial uses a more stable version of tensorflow which is TensorFlow Mobile, so follow the steps on how to implement your models and make them Optimized for Mobiles.

We are going to make an Image Classifier by Retraining the Final (Bottleneck) Layer of the Inception-v3 model and then Optimize the model for your smart devices.

The tutorial contains only 5–6 steps:

Step 1: Create your Model with TensorFlow

I’m pretty sure you already know this step, since you are learning to run the same model on the smartphones.

Also to keep this tutorials strictly focused on the Implementing models on the smartphones, I would recommend this quick tutorial 🔹 Train Inception with Custom Images on CPU 🔹 So we’d be on the same page 📄 and you can start the things in a new directory with a newly trained Model.

FYI: The data 📗 on which the model is trained contains types of Flowers which are Tulips 🌷, Daisy, Sunflower 🌻, Dandelion and Roses 🌹

And after this you should have these two files:

tf_files/retrained_graph.pb 

which contains a version of the selected network with a final layer retrained on your categories.

and

 tf_files/retrained_labels.txt

which is a text file 📝 containing labels .

Step 2: Create an Optimized Model 😎

Run the following command prompt (keep the path correct)

python -m tensorflow.python.tools.optimize_for_inference \
  --input=tf_files/retrained_graph.pb \
  --output=tf_files/optimized_graph.pb \
  --input_names="input" \
  --output_names="final_result"

It will create a new optimized model file tf_files/optimized_graph.pb

Note: If you are getting an error KeyError: “The following input nodes were not found: {‘input’}\n” then change "input" to "Mul".

Trade-Offs 😇

To reduce the pre-processing of an app and to decrease the size of the library at the same time, tensorflow only supports the subset of operations that are commonly used during inference. The operations which are not supported are tensorflow/contrib/makefile/tf_op_files.txt

Now, just to make sure that whatever graph file we’ve just created includes the supported operations as follows…

Verify ✔️

To make sure that your new optimized graph is running and the optimize_for_inference file that removes all nodes that aren’t needed for a given set of input and outputs and hasn’t altered the output of the network.

Compare the output of the same image using the label_file on both the graph retrained_graph.pb with optimized_graph.pb

with retrained_graph.pb

python -m scripts.label_image \
  --graph=tf_files/retrained_graph.pb \
  --image=tf_files/flower_photos/daisy/3475870145_685a19116d.jpg

with optimized_graph.pb

python -m scripts/label_image \
    --graph=tf_files/optimized_graph.pb \
    --image=tf_files/flower_photos/daisy/3475870145_685a19116d.jpg

Run these commands one by one and if both the outputs are identical that means optimized_graph.pb is created perfectly 🎉

Step 3: Quantize the Model, then Compress

The issue is still that the size of the model is still big and definitely not suitable for mobiles. Since, the majority of the space taken up by the graph is by the weights, which are large blocks of floating point numbers. Each weight has a slightly different floating point value, with very little regularity.

But compression works by exploiting regularity in the data, which explains the failure here.

Quantization helps to reduce the size of the neural network by factors, by performing quantization on the weights of the net. This gives a lot more repetition in the graph and helps significantly in the compression afterwards.

Now use the quantize_graph script to apply changes to the graph:

python -m scripts.quantize_graph \
  --input=tf_files/optimized_graph.pb \
  --output=tf_files/rounded_graph.pb \
  --output_node_names=final_result \
  --mode=weights_rounded

and now Compress the Model:

gzip -c tf_files/rounded_graph.pb > tf_files/rounded_graph.pb.gz

gzip -l tf_files/rounded_graph.pb.gz

This will create a rounded_graph.pb file.

You should see some significant improvement in the compression.

Note: If you are getting any error when running the quantize_graph download this file and paste it in the tools/quantization /quantize_graph.py in the tensorflow library (wherever TensorFlow is installed).

From here on the tutorial is divided into two sections Android and iOS.

🍎 iOS 📱

Step 4: Add TensorFlow-experimental pod

Add TensorFlow-experimental pod to your pod file, which installs a universal binary framework. This is the most easiest way to run tensorflow on iOS.

Step 5: Create your App

🔹 Create your own app or load your already-created app in XCode.

🔹Add a file named Podfile at the project root directory with the following content:

target 'YourProjectName'
pod 'TensorFlow-experimental'

🔹 Run pod install to download & install the TensorFlow-experimental pod.

🔹 Open YourProjectName.xcworkspace and add your code.

🔹 In your app’s Build Settings, make sure to add $(inherited) to the Other Linker Flags, and Header Search Paths sections.

Step 6: Running Samples

You’ll need Xcode 7.3 or later to run our iOS samples.

There are three examples in simple, benchmark, and camera. You can clone the code.

Also, download Inception v1 from root of tensorflow and extract the label and graph files into the data folders inside both the simple and camera examples using these steps:

mkdir -p ~/graphs
curl -o ~/graphs/inception5h.zip \
 https://storage.googleapis.com/download.tensorflow.org/models/inception5h.zip \
 && unzip ~/graphs/inception5h.zip -d ~/graphs/inception5h
cp ~/graphs/inception5h/* tensorflow/examples/ios/benchmark/data/
cp ~/graphs/inception5h/* tensorflow/examples/ios/camera/data/
cp ~/graphs/inception5h/* tensorflow/examples/ios/simple/data/

Change into one of the sample directories, download the Tensorflow-experimental pod, and open the Xcode workspace. Note that installing the pod can take a long time since it is big (~450MB). If you want to run the simple example, then:

cd tensorflow/examples/ios/simple
pod install
open tf_simple_example.xcworkspace #note .xcworkspace,not .xcodeproj
                                     

Run the simple app in the XCode simulator. You should see a single-screen app with a Run Model button. Tap that, and you should see a Grace Hopper image. Once you build and run that, you should get a live camera view that you can point at objects to get real-time recognition results.

Note: I’m pretty sure that I’ve done some mistake or left something on the iOS section. Please go through the following official links and please comment down below if you get any error, the community will help you.

Now, you can skip the Android section of this article.

https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/ios#building-the-tensorflow-ios-libraries-from-source

Building TensorFlow on iOS | TensorFlow

🍭 🍦 Android 🍞🐝 🍩

Step 4: Setup Android Studio and Test Run

There are two ways to do this Android Studio and Bazel. I’ll be using AS since more people are familiar with it.

If you don’t have it installed already, go here and install it

Download Android Studio and SDK Tools | Android Studio

Test Run 🏃

Just to check whether everything is working correctly in Android Studio, let’s do a test run.

🔸 Open Android Studio and select “📁Open an existing Android Studio project”.

🔸 Go to tensorflow-for-poets-2/android/tfmobile directory.

🔸 Open Build.gradle file and sync the Gradle. If everything works perfectly, click BUILD>BUILD APK button.

Note: If you are new to Android Studio like me and having issues with something, comment below.

Now a folder should come with the app.apk file, copy it in your phone and install it. Also, keep in mind to turn on the Developer Mode in your phone.

Step 5: Run Customized App 👏 👏 👏

If everything works fine till here, it’s a cakewalk from now.

The Default app is an classify images app with 1000 categories from Imagenet.

Now, to run our customized app perform the following two steps:

Add your model files to project

Right now the demo app is looking at the graph.pb file and label.txt file present in the android/tfmobile/assets not at your rounded_graph.pb and retrained_labels.txt

Now, replace the files with the following command or you can do it manually.

cp tf_files/rounded_graph.pb android/tfmobile/assets/graph.pb
cp tf_files/retrained_labels.txt android/tfmobile/assets/labels.txt

Change the "output_name" in ClassifierActivity.java file

The output node for our model has a different name: "final_result". Open ClassifierActivity.java and update the OUTPUT_NAME variable as follows:

private static final String INPUT_NAME = "input";
private static final String OUTPUT_NAME = "final_result";

👏 👏 👏 RUN again and things should work now. 👏 👏 👏

Note: If you get any error or hit any wall comment below.

️I have tried to keep the article as exact and easy to understand as possible. Any comments, suggestions or if you have any queries, write it in the comments.

For further Tutorials on how to use TensorFlow in Mobile apps follow me on Medium, Facebook, Twitter, LinkedIn, Google+, Quora to see similar posts.

Clap it! Share it! Follow Me!

Happy to be helpful. kudos…..

Previous stories you will love:

1. TensorFlow on Mobile: TensorFlow Lite

2. Epoch vs Batch Size vs Iterations

3. TensorFlow Image Recognition Python API Tutorial On CPU

4. Activation Function Explained: Neural Networks

How to Send Emails using Python