The world’s leading publication for data science, AI, and ML professionals.

Artificial Intelligence Without Labeled Data

Let's talk about Unsupervised Deep Learning AI, and Transfer Learning.

Let’s talk about Unsupervised Deep Learning AI, and Transfer Learning.

In past articles, I have put a strong emphasis on the many benefits of supervised learning. When you have labeled data, machine learning works. It’s great. Now, what about cases where you don’t have the labeled data? It’s a good idea to get some labelled data, but what else can we do?

A wise meme indeed
A wise meme indeed

Well, buckle up. There are two strategies we use when data for supervised learning is not readily available: transfer learning, and unsupervised learning.

Transfer learning, where you benefit from someone else’s already trained model, and customize it to your purposes, is pretty cool. A pre-trained image recognition Convolutional Neural Network (CNN) like VGG-19 or inception v3 can be re-trained to recognize new stuff. I showed CNN examples in [[this article](https://medium.com/@lemaysolutions/ai-is-this-porn-2695c5e6fa47)](https://medium.com/@lemaysolutions/machine-learning-in-medicine-demo-time-ffd17ed8a72a), this article, and this article, but more recently I prefer to use keras (high level) rather than tensorflow (lower level). Word embedding models like the Google News word2vec model, and the Wikipedia word2vec model are quite popular for analyzing text without labeling the data.

We talked about transfer learning for images and text, what else can we do?

Unsupervised learning goes one step further than transfer learning, and has the model "learn" from the unlabeled data. I wrote a paper earlier this year on unsupervised learning from images AND text at the same time.

I want to tell you a quick trick I used for that work, to give you a sense of how you can learn something from unlabeled text data.

So let’s say you have to watch a user and learn from their onscreen actions. Super complicated topic. Read the paper if you are curious about the larger solution. For this article, let’s narrow our focus to just the onscreen text part of the problem, assuming the spelling is correct from the OCR. The challenge in this simplified problem is to recognize the meaning of onscreen text where the tokens are often not present in pre-trained word2vec models like the Google News vectors. The model has 3 billion words in it, which seems like a lot, but not many of the interesting words you may see on a computer screen, like "NullPointerException". So, what do we do?

Solved! We can build a new word embedding model from scratch using the text we have seen on the screen, and eventually, after watching the screen text change for a long time, we will have a big enough corpus to "understand" these onscreen words in the context that they appear. So that’s a nice start, but we benefit even more from extracting additional documents from the internet that use the words we saw on the screen, and including them in our corpus. So, for example, if we see "NullPointerException" and "BaseException", we can call Google Custom Search Engine (CSE), and get documents that have both of these terms (keywords). Hopefully those documents put these keywords in more context. We add the new documents to the corpus of stuff we saw on the screen, hopefully gaining new insight in our embedding model. I ended up doing this for triplets of keywords as well.

This is a high level view of the overall system for learning to understand the computer screen. It's a bit beyond the scope of this article. The keyword clustering block was the unsupervised text understanding part of the system.
This is a high level view of the overall system for learning to understand the computer screen. It’s a bit beyond the scope of this article. The keyword clustering block was the unsupervised text understanding part of the system.

If you enjoyed this article on Artificial Intelligence, then please try out the clap tool. Follow us on medium. Go for it. I’m also happy to hear your feedback in the comments. What do you think?

Happy Coding!

-Daniel [email protected] ← Say hi. Lemay.ai 1(855)LEMAY-AI

Other articles you may enjoy:


Related Articles