Intelligent, realtime and scalable video processing in Azure
1. Introduction
In this tutorial, an end to end project is created in order to do intelligent, realtime and scalable video processing in Azure. In this, a capability is created that can detect graffiti and identify wagon numbers using videos of trains. Properties of the project are as follows:
- Intelligent algorithms to detect graffiti and identify wagon numbers
- Realtime and reliable way of processing videos from edge to cloud
- Scalable for exponential growth of number of videos
- Functional project that can be optimized to any video processing capability
The architecture of the project can be depicted as follows:
In this blog this architecture is realized as follows:
- 2a. Cognitive Services to detect graffiti on trains (Custom Vision)
- 2b. Cognitive Services to identify wagon numbers (Computer Vision OCR)
- 3. Azure Functions for parallel processing of videos
- 4. Power BI for visualization (optional)
- 5. IoT Edge architecture for auto-tiering data(optional)
- 6. Conclusion
In this blog, all video processing is done in Azure. Refer to this follow-up tutorial in which the graffiti detection is done on the camera (edge) itself. In the next chapter, Azure Cognitive Services will be deployed.
2. Azure Cognitive Services
Azure cognitive services are a set of APIs that can be infused in your apps. It contains intelligent algorithms for speech recognition, object recognition in pictures and language translation. The models are mostly pretrained and can be integrated “off the shelf” in your project. Most models can also be deployed as container on the edge. In this project, two APIs will be used:
- Custom Vision that will be used to detect graffiti on trains. This model needs pictures for trains with/without graffiti to learn. This step can be seen as “adding the last custom layer in the neural network of an image recognition model that was already trained in Azure Cognitive Services”
