The world’s leading publication for data science, AI, and ML professionals.

Linear Algebra: Euclidean Vector Space

Part 5: A Gentle Introduction to Euclidean Vector Space

Photo by Karsten Würth on Unsplash
Photo by Karsten Würth on Unsplash

Introduction

Most of the time in both machine learning and deep learning, we are working with vectors. And the vector space model can represent the relationship between data as vectors. Moreover, from a geometric perspective, it is also able to compare the similarity of two vectors either using the distance between the two vectors (euclidean distance) or the angle between two vectors (Cosine Similarity).

Vector

Let’s begin with the geometry of the vector in 2-space.

Image 1. Example of vector in 2-space. (Image by Author)
Image 1. Example of vector in 2-space. (Image by Author)
  • Two vectors u = (u₁, u₂) and v = (v₁, v₂) are equal if u₁ = v₁ and u₂ = v₂.
  • The sum of vector u and v is defined as u + v = (u₁ + v₁, u₂ + v₂)
Image 2. Example of the sum of vectors. (Image by Author)
Image 2. Example of the sum of vectors. (Image by Author)
  • The scalar k multiplies with vector u is defined as ku = (ku₁, ku₂)
Image 3. Example of the scalar multiplies with vector. Image by Author.
Image 3. Example of the scalar multiplies with vector. Image by Author.
  • The negative of vector, -v, is defined to be the vector that has the same magnitude as v but in the opposite direction.
  • The difference of vector is defined as u-v = u + (-v)
Image 4. Example of the difference of vectors. (Image by Author)
Image 4. Example of the difference of vectors. (Image by Author)

Norm and Distance

  • The length of a vector is often called the norm.
Image 5. The length of a vector, norm. (Image by Author)
Image 5. The length of a vector, norm. (Image by Author)
  • Distance between 2 points is defined as follows:
Image 6. Distance between 2 points. (Image by Author)
Image 6. Distance between 2 points. (Image by Author)

Euclidean n-space

  • If n is a positive integer, there is a sequence of n real numbers v₁, v₂, …, vₙ then we write: v = (v₁, v₂, …, vₙ)
  • The set of all vectors with n components is called Euclidean n-space and is denoted as Rⁿ.

Properties of vectors in Rⁿ

  1. u + v = v + u
  2. u + (v + w) = (u + v) + w
  3. k(u + v) = ku + kv
  4. (k + m)u = ku + mu
  5. u + 0 = 0 + u = u
  6. u + (-u) = 0
  7. 1u = u where u, v, w are vectors and k, m are constants

Euclidean inner product

  • If u and v are vectors in Rⁿ, the Euclidean inner product is defined as u . v = u₁v₁ + u₂v₂ + … + uₙvₙ

And it follows:

Image 7. Proof of Cauchy-Schwarz inequality. (Image by Author)
Image 7. Proof of Cauchy-Schwarz inequality. (Image by Author)
Image 8. Proof of u.v=1/4||u+v||² -1/4||u-v||²
Image 8. Proof of u.v=1/4||u+v||² -1/4||u-v||²
  • Properties: u.v = v.u (u + v).w = u.w + v.w (ku).v = k(u.v) v.v ≥ 0, v.v = 0 only if and only if v = 0 where u, v, w are vectors and k is constant

Norm and Distance in Rⁿ

  • The norm in Rⁿ is defined as:
Image 9. The norm in Rⁿ. (Image by Author)
Image 9. The norm in Rⁿ. (Image by Author)
Image 10. Euclidean distance between 2 points in Rⁿ. (Image by Author)
Image 10. Euclidean distance between 2 points in Rⁿ. (Image by Author)
  • Properties: ||u|| ≥ 0, ||u|| = 0 if and only if u = 0 ||ku|| = |k| ||u|| ||u + v|| ≤ ||u|| + ||v|| d(u, v) ≥ 0, d(u, v) = 0 if and only if u = v d(u, v) = d(v, u) d(u, v) ≤ d(u, w) + d(w, v) where u, v, w are vectors and k is constant

Angle in Rⁿ

If u and v are vectors in Euclidean n-space, the angle between u and v (θ) is defined as:

The derivation of cosθ can refer to Image 7.

Image 11. The angle between 2 vectors in Rⁿ. (Image by Author)
Image 11. The angle between 2 vectors in Rⁿ. (Image by Author)

Recommended Reading

Linear Algebra: Systems of Linear Equations and Matrices, with Python

Linear Algebra: Matrix Operations and their Properties, with Python

Linear Algebra: Finding Inverse Matrix, with Python

Linear Algebra: LU Decomposition, with Python

Linear Algebra: Orthogonal Vectors

Linear Algebra: General Vector Space

Linear Algebra: Discovering Eigenvalues and Eigenvectors for Diagonalization


References

[1] Vector Space – Wikipedia

[2] Euclidean Space – Wikipedia

[3] National Chung Cheng University Lecture – Wei-Ta Chu, Euclidean Vector Space, 2008


Related Articles