5 lessons learned at North Star AI
This week I went for a trip to Tallinn, Estonia. What was most exciting about the city this week was not it’s beautiful old town, intense winter or mild-mannered people, however — It was the happening of North Star AI, a machine intelligence conference for developers.
The speaking schedule was populated by champions of AI & CS such as Travis Oliphant (creator of NumPy), Sayan Pathak (principal ML scientist at Microsoft) and Ahti Heinla (co-developer of Skype and co-founder of Starship) among many others.
I came with a desire to explore new perspectives, meet smart people and learn from the best — You might have seen me there, eagerly taking notes. In this article I will attempt to distill five lessons taken home from the experience. These are patterns of commonality in topics discussed by the speakers that stimulated valuable insights for me. Be advised that these lessons do not necessarily reflect the views of the speakers, rather my interpretation and aggregation of their points. Let’s get started!
1: AI has become feasible for business
The conference participants themselves were evidence of the new practical feasibility of AI technologies in actual products. You would hear about its application in bigger companies such as Starship (who make delivery robots), Taxify (who connects passengers with drivers) and Elisa (a telecom provider who uses AI to create chatbots that improve customer service) but also for a number of startups and mid-sized companies. For many startups in tech, AI seems to be part of the core offering and in bigger companies, the technology is often being considered for improved customer service and business intelligence.
A reason for this development is the reached maturity of a number of machine learning frameworks such as Tensorflow, CNTK and PyTorch. Another very prominent reason, as highlighted by Travis Oliphant, are the high-level APIs that are becoming available which allow us to more quickly sketch out and evaluate models such as Keras, Azure ML, Google Cloud ML Engine and AWS Machine Learning. These APIs make it really practical to start building machine learning models at a level of abstraction where upfront cost of development balance better with added business value.
An element of progress in making AI more practical might also be standards for intermediary representations (IR) such as ONNX (Open Neural Network Exchange): An open format for deep learning models which makes interop between frameworks possible. This idea was stressed by both Travis and Sayan Pathak. Using ONNX, you could use CNTK to deploy a deep learning model which was originally constructed with PyTorch or Caffe, for example. This is particularly exciting for someone like me who prefers to work with .NET in production and, I imagine, for many businesses looking to integrate deep learning into their existing stack. ONNX is already supported natively in many frameworks and has converters for some of the rest, like Tensorflow.
2: Multi-agent systems yield great power
Towards the end of the day, Sean Gourley, CEO at Primer, gave a really interesting presentation of their product: A machine learning system to automate analysis of large textual datasets and generate natural language reports used by companies such as Walmart. I can recommend checking out their website and reading the reports — The results are really quite impressive.
To achieve this ambitious goal, Primer makes use of six interacting intelligence engines — machine learning systems to perform different tasks. One engine serves the purpose of identifying entities and structured data while another looks for events by clustering entity relationships as a function of time. These agents serve different purposes and can be used in cooperation to solve the bigger problem of performing an exhaustive topic analysis. Finally, their story engine puts together high-dimensional topic and event information and generates a natural language report.
This way of thinking really opened my eyes about how complex problems can be approached. In terms of architecture, multi-agent systems that are made to interact and that can be recombined in different ways seem incredibly powerful. It divides a big problem into smaller domains and allow for a kind of group interaction within the system while making individual modules easier to test and update. It’s like having a diverse team of people working together!
Maxim Orlovsky, founder of Pandora Boxchain, also drew attention to the great power of multi-agent systems. In a practical sense, we will soon find ourselves in a world of interacting agents like self-driving cars on the road. But as the complexity of this ecosystem of independent agents scale up, how can we protect ourselves from malicious instances or terrorists? Maxim stressed that we should start thinking about byzantine fault tolerant systems for managing interacting agents — That means working towards making reliable systems from unreliable parts.
Pandora Boxchain is a decentralised blockchain-based AI market in which machine learning models can be executed by a network. Within a platform like this, network participants with rational self-interest can protect each other against malicious agents, incentivized by decentralized economics and game theory rather than governmental regulation. It is a really interesting and visionary project that I will look forward to following in the future.
3: AI is not a silver bullet
A challenge of communication that I have met when discussing the implementation of AI in business is the tendency of thinking that it, by itself, will do something extraordinary for the company even though an actual problem to be solved has yet to be defined. I think this is a common pattern with new technologies as hype tends to spread faster than understanding.
Habib Adam, data scientist at Transferwise, articulated this really well: He proposed that we should remember to be problem driven rather than solution driven. Instead of attempting to find “something to do” with AI, we should consider which problems we are experiencing and which kinds of insights might be useful to us. He highlighted three important factors at play when considering a machine learning solution for a problem:
1: Randomness. Do we think there is a pattern to be found here?
2: Related data. Which (and how much) relevant data do we have available?
3: Ability to act. Which useful actions can we actually take from this data?
Habib further explained that when we’ve actually solved a problem, we can start thinking about how our solution can be made more flexible and extended to other domains.
To be fair, understanding how to define these problems can be challenging and takes practice. In order to counteract misunderstanding, we need to educate decision makers in business so they can take appropriate action. Markus Lippus, co-founder of AI consultancy MindTitan, shared some great points for facing that issue. When collaborating with a company on AI or data science integration, they host workshops. Within these, there is a focus on problem discovery and figuring out pain points for the company — A problem driven approach. When the necessary understanding has been gathered, the MindTitan team can properly advise the collaborator on where and how AI solutions can deliver value to their business.
4: Process management is critical in scaling AI
As we begin to apply AI/ML solutions in bigger ways, process is becoming more important than ever. This is both due to more extensive systems but also the growing data science teams that are building them. Andrus Kuus, software analyst at Proekspert, alluded to the fact that much of the coming workforce in AI and data science will be students coming directly from university (which are often inexperienced in teamwork) and that we can benefit from putting practices into place to enhance team synergies. He highlighted a number of perspectives that he had discovered when growing their data science team:
1: Choose tools and practices that will help you achieve clarity.
2: Ask the right questions and validate ideas before trying solutions.
3: Discuss regularly.
While I cannot remember his advice word for word, I believe that is the gist of it. In a world that is rapidly changing, a critical approach with focus on clarity and internal communication definitely seems like a great approach.
Additionally, we need to remember to plan ahead for larger scale AI systems. Travis Oliphant hit an important point here: Do you have a plan for when your model goes stale? It definitely will, it’s just a matter of time. The world changes all the time. So does data and the way we interact with it. As such, you should make sure you have a plan in place for maintaining and updating your model as time passes. Travis calls this process model management.
When models are trained, some trade-offs about decision boundaries happen on the basis of data distribution. If you are in the unfortunate position where your model has “gone stale” or the distribution of input data has changed and retraining is not an option, Peter Prettenhofer, data scientist at DataRobot, suggest looking into dataset shift. This concept involves using techniques like statistical distance and importance re-weighting to understand how your data distribution has changed and correct for it.
5: Nature is a great teacher
Antti Rasmus, CTO at Curious AI, delivered a really interesting presentation about the work they have been doing in modeling human imagination. He drew a parallel to Daniel Kahneman’s “Thinking, Fast and Slow” which presents a dichotomy in human thinking split in two systems: System 1 for fast and intuitive thinking and system 2 for deliberate planning and imagination. In autonomous vehicles, examples of such behavior could be the combination of staying on the road (system 1 which is intuitive and in the moment) while also planning for possible future outcomes like what to do if a pedestrian or another vehicle behaves erratically (system 2, planning and imagination). Curious AI had achieved good estimations of this with what they call model-based reinforcement learning — They demoed this with a spider-like digital creature which gradually learned to walk in an unstructured way.
Maxim Orlovsky also highlighted this perspective in his visionary talk: In the future, we might have great benefit in further modeling biological systems. In a sense, the ideas for decentralization with Pandora Boxchain could also be conceived as introducing a sort of human-like group dynamic to multi-agent AI systems.
These presentations struck a chord with me, as I too believe there is much potential waiting to be unleashed from understanding and modeling human systems. My area of research is drawing on cognitive science for modeling the human auditory system in order to create better AI models for audio signal processing. So far, this perspective has been really powerful for me and have resulted in new ideas and a more guided approach to the problem domain. If you’re interested, you can read a primer on my project here: “The promise of AI in audio processing”.
Those were the five most valuable take-aways I got from the North Star AI conference. I should extend a great thanks to the organizers and speakers who have certainly broadened my horizon and made way for new ideas.
If this recap was helpful to you, go ahead and leave a few claps.
