Do it for the ‘gram: Instagram-style Caption Generator
Generating captions for Instagram photos using a Keras CNN-RNN framework
By Camille Bowman, Sejal Dua, and Erika Nakagawa
1. Introduction
Image captioning refers to the Deep Learning application of generating a textual description of an image using Natural Language Processing (NLP) and Computer Vision (CV). This task requires an algorithm to not only understand the content of the image, but also to generate language that connects to its interpretation. We wanted to take this challenge one step further by generating captions specifically for Instagram pictures.
How would you describe the picture below?
A simple caption generator could describe the image as something along the lines of “four friends in a pink room”. Microsoft’s CaptionBot answered with “I think it’s a group of people posing for a camera”. Isn’t that how you would describe it as well? But if you wanted to post this image on Instagram, would that be the caption for it?
Instagram captions tend to be more advanced than a simple descriptor and consist of puns, inside jokes, lyrics, references, sentiment, and sarcasm. In some cases, the caption may not be relevant to the presented image at all. In this case…