The world’s leading publication for data science, AI, and ML professionals.

Eigenvectors & Eigenvalues – How to explain them to a 10-year-old

Eigenvalues & Eigenvectors are central (but not limited) to many of the well known machine learning algorithms. Algorithms like SVD, PCA…

Making less intuitive concepts intuitive

Photo by Benjamin Lizardo on Unsplash
Photo by Benjamin Lizardo on Unsplash

Eigenvalues & Eigenvectors are central (but not limited) to many of the well-known machine learning algorithms. Algorithms like SVD, PCA, spectral clustering, image segmentation, 3D reconstruction use eigenvalues & eigenvectors as their core logic to function.

Despite their usefulness, their criticality in many well-known applications & being part of linear algebra 101, they are often considered difficult to understand & even feared upon by budding statisticians/data scientists primarily due to how non-intuitive the concept can be.

If you’re hearing the word eigenvalue & eigenvector for the first time, the story will help to understand the core concept behind Eigenvectors & eigenvalues. If you know it well and perhaps want to explain this to your niece or nephew (or someone who is non-stats-savvy), this story will help with that 🙂

The story goes like this:

In a parallel universe, there lived two powerful forces – the matrices & the vectors. The matrices were bullies & vectors were timid & easily frightened. Whenever a vector meets a matrix, it would re-direct its path. If the vector was going for grocery shopping & meets a matrix in its path, it would turn around & go to his friend’s house. If it was going to the university & met a matrix in its path, it would turn to go to the mall.

Now like any other story, there were also rebels among the vectors. They were called the eigenvectors. They preached non-violence & were very stubborn on their cause. No matter the matrix, they always kept to their path. Not worried about being bullied, they kept going towards their destination for they were goal oriented. Yes, at times it would get difficult when a very powerful matrix would block their path & try to re-direct them. But it would only slow them down & not re-direct them. If a weak matrix would block their path, they would speed up further blowing the matrix away. This power of being adaptive to a strong or weak matrix was only possessed by the eigenvectors & they are called eigenvalues. Each matrix has its special troublemaker(s) aka their own eigenvectors.

what we learned: eigenvectors are special vectors that maintain their direction despite bullying by the matrices (in mathematics, we call this a matrix transformation). These vectors speed up (elongate) or slow down (shrink/squish) depending on the intensity of bullying (transformation) by a factor of eigenvalue.

In mathematical terms, the eigenvector-eigenvalue equation is denoted as:

Eigenvector & Eigenvalue equation
Eigenvector & Eigenvalue equation

Where A is the matrix, v denotes the corresponding eigenvector to the matrix & lambda denotes the eigenvalue. Another way to read the equation (if you want to sound smarter) is whenever a matrix A is multiplied by a special vector v, the vector gets scaled by a factor of lambda.

Now if you observe the above equation, the lambda (or the scaling factor), can be any number (including complex). If it’s positive, it means it’s pointing to the same direction as the original vector. There are also special boss matrices called singular matrices, where lambda is 0 and there are also matrices that have negative lambda which means the eigenvector reverses its direction but stays on the line it spans on.

The importance & application of Eigenvectors & Eigenvalues are more dependent on the basis-change & are beyond the scope of this article. If you’d like to learn more about why Eigen-things are so widely used in linear algebra & Machine Learning & how it simplifies certain calculations by order of 100 or more (under certain scenarios), please leave a comment 🙂

Like my article? Buy me a coffee

sundaresh is creating data science-related articles and loves teaching


Related Articles