Beyond tags and entering the semantic search era on images with OpenAI CLIP
A Google Colab code notebook exploring a real-world use case with OpenAI CLIP
Problem Statement:
Imagine that you are a writer and you are searching for the best image that goes with your blog or book. You have a search phrase in mind like “Tiger playing in the snow”. You go onto copyright-free image websites like Pixabay or Unsplash and try out various combinations of keywords like “Tiger”, “Snow”, “Tiger Snow” etc to find relevant images.
If you are lucky you find the exact image that you are looking for on the first page or in the top N retrieved results.
Since the images in these websites have only tags, you are limited by the results retrieved by tags and you need human supervision to further filter the most relevant images for your search phrase “Tiger playing in the snow”.
This is a problem if you are looking to find relevant images for your search phrases frequently and it is time-consuming to further filter on the images retrieved from tags.
What you need is a semantic search on top of the retrieved images with tags.
