AWS SageMaker: AI’s Next Game Changer
Today AWS SageMaker was released, and it is awesome. I have mentioned in previous articles that we do mostly AWS deployments for our clients. Smaller models fit in a DigitalOcean droplet, but CNNs and word embedding models really need GPU and lots of RAM. This is going to be another technical article to show you why today’s release was a big deal.
Let’s take a quick dive into the shiny new SageMaker service to get a sense of what it can do. This is the tip of the iceberg.
Step by step getting a notebook running:
First log into your AWS console:
Next, hop over to the SageMaker quickstart page to make a Notebook instance:
Let’s run the notebook itself on an ml.t2.medium instance. Under IAM role, use the wizard to make a role that can access your data. Of course, define an encryption key.
Once your notebook is created, simply click the open action, and the familiar jupyter notebook environment appears.
Here is what you should see:
To verify in your brain that the system is secure, simply try and go to the notebook URL in an incognito window and you will see that the session is not authenticated. Yep. Good to proceed.
In the sample-notebooks folder you have a bunch of examples. Make sure you have an S3 bucket to connect to these notebooks that the IAM role you defined up above has access to. Our bucket name is “lemaysolutions-demo”, and so in the notebook “notebooks/sample-notebooks/sagemaker-python-sdk/1P_kmeans_highlevel/kmeans_mnist.ipynb” change the bucket reference to the right name of your S3 bucket. This model is designed to recognize handwritten digits from the famous MNIST dataset from Yann LeCun.
OK. So run the first few cells and you should start to see results like this:
The data is being stored into your S3 bucket like this:
Note that the training instance is provisioned on demand from within the code (CRAZY!) and it is a ml.c4.8xlarge instance. It runs for a while so be patient.
The documentation indicates that the training job will take “between 7 and 11 minutes,” and in fact the timer on the cell indicates that the job took 6min 32s!
Setting up hosting for the trained model is also done programmatically. The docs indicate another 7 to 11 minute setup time, which turned out to be more like 12min 39s. Here is a hint. If there are dots in the favicon of the notebook, it is running.
Here is the output of the model on the dataset. In a very basic sense, these 10 clusters represent the result of a K-means classifier grouping these digits into 10 piles according to their similarity. By looking at the shapes you can intuitively tell the decent job this system did with low effort.
At the end of the day you should have created an S3 bucket, a notebook instance, a job, and a trained model.
And so, in conclusion, AWS SageMaker is wonderful. It’s going to change how we do our consulting completely. We are all about fast paced development and showing value early, and SageMaker helps us do that right now on day 1. Amazon didn’t even pay me to say it 😜.
If you liked this article on the brand new AWS SageMaker, have a look at some of my most read past articles, like “How to Price an AI Project” and “How to Hire an AI Consultant.” In addition to business-related articles, I also have prepared articles on other issues faced by companies looking to adopt deep machine learning, like “Machine learning without cloud or APIs.”
Happy Coding!
-Daniel
daniel@lemay.ai ← Say hi.
Lemay.ai
1(855)LEMAY-AI
Other articles you may enjoy:
