
Introduction
Most of the time in both machine learning and deep learning, we are working with vectors. And the vector space model can represent the relationship between data as vectors. Moreover, from a geometric perspective, it is also able to compare the similarity of two vectors either using the distance between the two vectors (euclidean distance) or the angle between two vectors (Cosine Similarity).
Vector
Let’s begin with the geometry of the vector in 2-space.

- Two vectors u = (u₁, u₂) and v = (v₁, v₂) are equal if u₁ = v₁ and u₂ = v₂.
- The sum of vector u and v is defined as u + v = (u₁ + v₁, u₂ + v₂)

- The scalar k multiplies with vector u is defined as ku = (ku₁, ku₂)

- The negative of vector, -v, is defined to be the vector that has the same magnitude as v but in the opposite direction.
- The difference of vector is defined as u-v = u + (-v)

Norm and Distance
- The length of a vector is often called the norm.

- Distance between 2 points is defined as follows:

Euclidean n-space
- If n is a positive integer, there is a sequence of n real numbers v₁, v₂, …, vₙ then we write: v = (v₁, v₂, …, vₙ)
- The set of all vectors with n components is called Euclidean n-space and is denoted as Rⁿ.
Properties of vectors in Rⁿ
- u + v = v + u
- u + (v + w) = (u + v) + w
- k(u + v) = ku + kv
- (k + m)u = ku + mu
- u + 0 = 0 + u = u
- u + (-u) = 0
-
1u = u where u, v, w are vectors and k, m are constants
Euclidean inner product
- If u and v are vectors in Rⁿ, the Euclidean inner product is defined as u . v = u₁v₁ + u₂v₂ + … + uₙvₙ
And it follows:


- Properties: u.v = v.u (u + v).w = u.w + v.w (ku).v = k(u.v) v.v ≥ 0, v.v = 0 only if and only if v = 0 where u, v, w are vectors and k is constant
Norm and Distance in Rⁿ
- The norm in Rⁿ is defined as:

- Euclidean Distance between two points in Rⁿ is defined as:

- Properties: ||u|| ≥ 0, ||u|| = 0 if and only if u = 0 ||ku|| = |k| ||u|| ||u + v|| ≤ ||u|| + ||v|| d(u, v) ≥ 0, d(u, v) = 0 if and only if u = v d(u, v) = d(v, u) d(u, v) ≤ d(u, w) + d(w, v) where u, v, w are vectors and k is constant
Angle in Rⁿ
If u and v are vectors in Euclidean n-space, the angle between u and v (θ) is defined as:
The derivation of cosθ can refer to Image 7.

Recommended Reading
Linear Algebra: Systems of Linear Equations and Matrices, with Python
Linear Algebra: Matrix Operations and their Properties, with Python
Linear Algebra: Finding Inverse Matrix, with Python
Linear Algebra: LU Decomposition, with Python
Linear Algebra: Orthogonal Vectors
Linear Algebra: General Vector Space
Linear Algebra: Discovering Eigenvalues and Eigenvectors for Diagonalization
References
[2] Euclidean Space – Wikipedia
[3] National Chung Cheng University Lecture – Wei-Ta Chu, Euclidean Vector Space, 2008