Member-only story
Machine Learning Does Not Only Predict the Future, It Actively Creates It
A primer on position bias (and why it matters)
Standard Machine Learning curricula teach that ML models learn from patterns that exist in the past in order make predictions about the future.
This is a neat simplification, but things change dramatically once the predictions from these models are being used in production, where they create feedback loops: now, the model predictions themselves are impacting the world that the model is trying to learn from. Our models no longer just predict the future, they actively create it.
One such feedback loop is position bias, a phenomenon that’s been observed in ranking models, those that power search engines, recommender systems, social media feeds and ads rankers, across the industry.
What is position bias?
Position bias means that the highest-ranked items (videos on Netflix, pages on Google, products on Amazon, posts on Facebook, or Tweets on Twitter) are the ones which create the most engagement not because they’re actually the best content for the user, but instead simply because they’re ranked highest.
This bias manifests because the ranking model is so good that users start blindly trusting the top-ranked…