This Social Sciences Theorem From 1785 makes for a Powerful Machine Learning Technique
It is not new that science borrows techniques from several fields to make new techniques. For example, genetic algorithms borrows knowledge from Darwin’s evolution theory and biology to make an algorithm that tackles hard problems. However, I will be discussing Genetic Algorithms in another post, since they are a fascinating topic themselves. Today I will discuss a technique from social sciences that involves voting.
Democracy and Machine Learning
Voting is the process of casting an opinion towards two or more options. That is, making a choice. When a group of voters cast their choices and the majority choice is selected as the group choice, we say that it was a democratic process (as long as voters were not influenced). Here’s where things get interesting. Suppose a scenario where there's only two choices: A and B. Moreover, suppose that there is one choice that is the correct. Finally, suppose that there's a probability that each voter will pick the right choice.
What I have just described can be found in Machine Learning as an ensemble method. An ensemble is a collection of Machine Learning models that take decisions as a group. It is the basis of bootstrap aggregation, also known as bagging [1].
It is demonstrated that bagging has several benefits over having a simple model. For example, having several weak models can reduce variance, decrease overfitting, increase stability and accuracy [1]. What is impressive is that this technique was actually proposed in 1785 by the Marquis de Condorcet, and is known as the Condorcet Theorem [2]. There’s one caveat: to obtain the benefits of bagging the probability of the weak models to make the right choice has to be larger than 50%. If the conditions of higher probability of making the right choice is not met, then the group decision is the same having only one individual making the choices [3].
I am writing daily knowledge pills so you can keep your knowledge sharp! I write about Data Science, Machine Learning in intersection with Finance. Feel free to connect and check out my other articles!
Reference
[1] Breiman L. Bagging predictors. Machine learning. 1996 Aug 1;24(2):123-40.
[2] Condorcet MD. Essay on the Application of Analysis to the Probability of Majority Decisions. Paris: Imprimerie Royale. 1785.
[3] De Prado ML. Advances in financial machine learning. John Wiley & Sons; 2018 Feb 21.
