EduArtha Learning Series

Mathematics for AI

Everything in AI is math. Without this, you cannot understand why any algorithm works. This book covers the four pillars — Linear Algebra, Calculus, Probability, and Optimization — with rigorous discussion, Python code, and AI applications.

⏱ Estimated study time: 3–6 months  |  13 Chapters  |  50+ Exercises

Part I

Linear Algebra

The language of data and transformations

Chapter 1

Vectors & Vector Spaces

Learning Objectives

  • Understand vectors as ordered arrays of numbers and their geometric meaning
  • Master vector operations: addition, scalar multiplication, dot product, cross product
  • Learn about vector spaces, subspaces, basis, and dimension
  • Connect vector concepts to real AI applications

What is a Vector?

A vector is an ordered list of numbers representing a point or direction in space. In AI, every data point is a vector. An image is a vector of pixel values. A sentence is a vector of word embeddings. A customer profile is a vector of features.

v⃗ = [v₁, v₂, ..., vₙ] ∈ ℝⁿ

A vector in ℝ² has 2 components (2D plane), ℝ³ has 3 (3D space), and ℝⁿ has n components (n-dimensional hyperspace). In ML, n can be thousands or millions — a 224×224 color image is a vector in ℝ¹⁵⁰,⁵²⁸.

Vector Operations

Addition and Scalar Multiplication

u⃗ + v⃗ = [u₁+v₁, u₂+v₂, ..., uₙ+vₙ]     c·v⃗ = [c·v₁, c·v₂, ..., c·vₙ]
Python
import numpy as np

u = np.array([3, 4, 1])
v = np.array([1, -2, 5])

print("Addition:", u + v)          # [4, 2, 6]
print("Scalar mult:", 3 * u)       # [9, 12, 3]
print("Magnitude:", np.linalg.norm(u))  # √(9+16+1) = √26 ≈ 5.1
print("Unit vector:", u / np.linalg.norm(u))  # Normalize to length 1

Dot Product

The dot product measures how aligned two vectors are. It is the single most important operation in ML — used in every neural network layer, every linear model, every attention mechanism.

u⃗ · v⃗ = u₁v₁ + u₂v₂ + ... + uₙvₙ = ||u⃗|| · ||v⃗|| · cos(θ)

If the dot product is positive, vectors point in similar directions. If zero, they are perpendicular (orthogonal). If negative, they point in opposite directions.

Python
u = np.array([1, 2, 3])
v = np.array([4, 5, 6])

dot = np.dot(u, v)  # 1*4 + 2*5 + 3*6 = 32

# Cosine similarity — measures angle between vectors
cos_sim = np.dot(u, v) / (np.linalg.norm(u) * np.linalg.norm(v))
print(f"Cosine similarity: {cos_sim:.4f}")  # 0.9746 — very similar

# This is how search engines find similar documents!

Why This Matters in AI

Neural network layers compute y = Wx + b — that's a dot product of weights W with input x for every neuron. Word embeddings use cosine similarity to find that "king" is to "queen" as "man" is to "woman". Recommendation engines compute dot products between user and item vectors to predict ratings.

Cross Product (3D only)

The cross product produces a vector perpendicular to both input vectors. Used in computer graphics, 3D physics, and robotics.

u⃗ × v⃗ = [u₂v₃ - u₃v₂, u₃v₁ - u₁v₃, u₁v₂ - u₂v₁]
Python
u = np.array([1, 0, 0])
v = np.array([0, 1, 0])
cross = np.cross(u, v)
print("Cross product:", cross)  # [0, 0, 1] — perpendicular to both

Vector Spaces, Basis & Dimension

A vector space is a collection of vectors that is closed under addition and scalar multiplication. The basis is the smallest set of vectors that can represent every vector in the space through linear combinations. The dimension is the number of basis vectors.

In ℝ³, the standard basis is {[1,0,0], [0,1,0], [0,0,1]}. Any 3D vector can be written as a combination of these three. In ML, finding a good basis (like PCA does) means finding the most informative directions in your data.

Linear Independence

Vectors are linearly independent if no vector can be written as a combination of the others. If vectors are dependent, they carry redundant information. In ML, redundant features waste computation and can cause numerical instability.

Python
# Check linear independence via matrix rank
vectors = np.array([
    [1, 0, 0],
    [0, 1, 0],
    [1, 1, 0]  # This = vec1 + vec2 → dependent!
])
print("Rank:", np.linalg.matrix_rank(vectors))  # 2, not 3 → dependent

Exercises

Exercise 1.1: Compute the cosine similarity between u=[1,2,3] and v=[3,2,1]. Are they similar?

u·v = 3+4+3 = 10, ||u|| = √14, ||v|| = √14

cos(θ) = 10/14 ≈ 0.714 — moderately similar (angle ≈ 44°)

Exercise 1.2: Why is cosine similarity preferred over Euclidean distance for text documents?

Documents of different lengths have very different magnitudes. A 1000-word document has much larger vector magnitude than a 100-word document, even if they discuss the same topic. Cosine similarity only measures the angle (direction), ignoring magnitude, so it compares content regardless of length.

Exercise 1.3: Given vectors a=[2,1], b=[1,3], c=[5,5]. Is c a linear combination of a and b?

We need scalars α, β such that α[2,1] + β[1,3] = [5,5].

2α + β = 5 and α + 3β = 5. From equation 1: β = 5-2α. Substituting: α + 3(5-2α) = 5 → α + 15 - 6α = 5 → -5α = -10 → α = 2, β = 1.

Check: 2[2,1] + 1[1,3] = [4,2] + [1,3] = [5,5] ✓. Yes, c = 2a + b.

Chapter Summary

  • Vectors represent data points in n-dimensional space; every ML input is a vector
  • Dot product measures similarity and is the core operation in neural networks
  • Cosine similarity compares direction (meaning) independent of magnitude
  • Linear independence ensures features carry non-redundant information