The world’s leading publication for data science, AI, and ML professionals.

Labelling instagram photos with Google Vision API

Today I will analyze instagram pictures using Google Cloud Vision

Today I will analyze instagram pictures using Google Cloud Vision

Methods

There are mainly two technical steps involved:

  • Scrap photos from my Instagram account using instaR package
  • Label the photos with Google Vision API in R via RoogleVision

As the how-tos on RoogleVision and instaR have already been covered [here](https://bigdataenthusiast.wordpress.com/2016/03/22/exploring-instagram-api-using-r/) and here, both of which involves setting up developer account and authenticate, I’ll focus on the experience of using them and the result.

Observations

One thing I learnt is that instagram only allows scraping of 20 latest photo in sandbox mode, unless you submit the app for review. Also if you’re using ‘carousel’ mode which lumps several photos in one post, instaR will download only the cover picture. Since I don’t instagram that often, that kinda covered the past 3 months’ pics and would grant me a decent understanding of what I’ve been posting.

Results

labelled instagrams
labelled instagrams

Some observations:

  • The api correctly labeled most photos except very abstract ones, like sand on beach (labeled as texture). Google Vision API focus more on labelling, color composition etc and doesn’t seem to offer captioning
  • The label detection algorithm will return several ranked results of the highest probabilities. Pictures with pre-dominated color block were labeled as the color more so than the object, i.e. human in red dress were labels as ‘red ‘as the first label and ‘facial expression’ as the second label
  • There is one complex photo left unlabelled. The photo contains a 3D-ish mural and food. Mural seems hard for the image recognition api to digest
  • I want to quantify what topics I photograph more in general, but given the 20 photo limit, I can only say I had repeat posts about flower and sky

This is #day45 of my #100dayprojects on Data Science and visual storytelling. Full code on my github. Thanks for reading. Suggestions of new topics and feedbacks are always welcomed.


Related Articles