Introduction
One of the most exciting projects I have been lucky enough to work on at Intel was leading the engineering team tasked with designing, implementing and deploying the software platform that enabled large-scale training based on Habana Gaudi® processors for MLPerf. I learned a lot on how to scale AI Training across a large hardware cluster, as well the challenges of building just building a data center. One thing that stood out was the immense amount of hardware, manual labor and power required to drive such a compute-intensive effort.
Modern AI/ML solutions have shown that given a large amount of computing resources, we can create amazing solutions to complex problems. Applications leveraging solutions such as DALL·E and GPT-3 to generate images or create human-like research papers are truly mind-blowing.
While I was on vacation (see picture above), I spent some time thinking about the explosion in model parameter size in the last few years and how compute accelerators have raced to keep up. I found this post, Large Language Models: A New Moore’s Law?, from the folks at HuggingFace that I found interesting, especially since they understand Deep Learning models much better than I ever will.
I found their comparison to Moore’s law, the supposition that transistor count and hence compute power doubles every two years, particularly appropriate. Another similar analogy that came to mind was how in the late 1990s and early 2000s, there was a CPU frequency scaling race. In both cases, compute hardware grew rapidly (even exponentially), but eventually began to run into the proverbial wall.
The HuggingFace article describes how to reduce the amount of compute required to bring a model to production. These are very important techniques, but I think just as important is finding alternative methodologies that more efficiently solve the same problems. Just like when CPU frequency reached a limit, the computer industry switched to focus on parallelism/instructions per cycle (IPC)/etc. The AI/ML work over the past decade has shown us some examples of what is possible, now the question is can we provide the same solutions using different, more optimized techniques.
Some fun contenders
Obviously, I don’t know all the research that is going on in this space, and I would never claim to, so here is a small list of interesting techniques that at least provide some promise in various areas. Rather than diving into a ton of depth and trying to re-create some of the great work people have done, I’m mostly trying to cover when we might use these technologies to solve problems that may also map to efforts in the AI/ML domain.
Similarity Search
Similarity search is an approximate, nearest neighbor (ANN) algorithm that can be used to identify groups of data that take as input a large vector. It takes this vector and an existing database of similar vectors and attempts to identify which vectors in the database are most like the input vector. The cool thing about this is that the vectors can be anything, so the applications of a similarity search can be broad. For example, it can be used to determine if some digitized assets (images/sounds/etc) are similar to other assets.
For a better description of how Similarity Search can be leveraged and how novel techniques can provide good accuracy with good power usage check out this post:
The Power of Intel® Hardware for Billion-Scale Similarity Search
Multimodal Cognitive AI
As the name suggests, multimodal cognitive AI is a field that is attempting to take solutions that exist in various spaces and logically combine them with each other to provide context. This opens the possibility of combining a lot of well-trained models together to provide better solutions. I also have been looking at this wondering, does this open the door for some smaller, less accurate, less compute-intensive models to be leveraged in solutions where they previously might not have had enough accuracy without context. Honestly, I’m not sure, and would love to hear from more domain experts on whether this idea makes any sense or not…
Interestingly, as I was going to publish this, Intel’s VL-InterpreT won a best demo award at the IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR) 2022 event! The tool helps us understand how Vision-Languange transformers behave.
Neuromorphic Computing
Neuromorphic computing is probably my favorite of all three of these approaches, maybe because it has a hardware component, or maybe because in my mind it is sort of another phase in AI/ML modeling.
I first learned about neural networks, one of the major building blocks of deep learning algorithms, when I was in college 20+ years ago. The concept made sense at the time, discrete layers passing information to each other with connections and mathematical operations within a node/layer. When described in the textbook as similar to how the brain works, I accepted it at face value.
The cool thing about neuromorphic computing is that it uses hardware to more accurately mimic how neurons in the nervous system work. Rather than having centralized shared storage and memory, a neuromorphic chip has several individual cores with their own storage and local memory. The cores are fully connected to each other, which allows for a myriad of communication flows between the cores. Also, the algorithms for a neuromorphic computing system may take into account a variety of input factors including the rate of input over time or inter spike intervals for example.
Given that it’s a relatively new approach to solving problems, it requires some new programming paradigms. Intel, as one of the leaders in the hardware research space, has open-sourced a software framework called Lava to help programmers leverage the hardware capabilities. Lava supports deep learning-like algorithms, as well as other types of algorithms.
One potentially huge benefit of neuromorphic computing is that this hardware can often provide solutions to some problems in the AI space at a fraction of the power cost. There’s lots of work to be done, but initial work in this area is very interesting and very promising. Additionally, neuromorphic computing can provide solutions to problems that are not easily solveable through traditional computing methods. It is not just an AI hardware optimization, but a completely new way to think about computing.
The first link here is a fantastic resource, they have a really nice overview and there were some workshops with some really easy-to-understand overview and deep-dive videos.
Conclusion
There are a lot of exciting areas of research in the AI/ML space that seem to lend themselves to creating more efficient solutions that require less computing power. There will always be a space for compute-intensive AI/ML, and a lot of that work informs the possibilities of outcomes the space can deliver.
However, as models get bigger and bigger and we reach the point of diminishing returns in terms of accuracy and trainability, new alternative approaches to solving similar problems that require less large-scale compute are becoming more mature.
Similarity search, multimodal cognitive AI and neuromorphic computing are examples of such approaches, I’d love to hear what other novel approaches I should check out, let me know in the comments or on Twitter!
Want to Connect?If you want to see what random tech news I'm reading, you can follow me on Twitter.Tony is a Software Architect and Technical Evangelist at Intel. He has worked on several software developer tools and most recently led the software engineering team that built the data center platform which enabled Habana's scalable MLPerf solution.Intel, the Intel logo and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
