Vectors and Matrices

A vector is an ordered list of numbers used as a point, direction, feature row, coefficient set, or embedding. A matrix is a rectangular array that can store a dataset, a batch of vectors, or a linear map applied through matrix multiplication.

Defining math

A vector $x\in {\mathbb{R}}^d$ has entries $x_1,\dots ,x_d$. A matrix $A\in {\mathbb{R}}^{m\times n}$ has entries $a_{ij}$ and maps $x\in {\mathbb{R}}^n$ to $Ax\in {\mathbb{R}}^m$:

$$(Ax{)}_i=\sum _{j=1}^n{a}_{ij}{x}_j.$$

The dot product

$$x^{\top }y=\sum _{j=1}^d{x}_j{y}_j$$

turns vectors into geometry: it defines angles, orthogonality, projections, and many norms and distances. In machine learning, the same object can be read as either data or parameters; a row of $X$ is a sample, while a column of $W$ in a neural layer is a learned direction.

Worked example

Let $v=(2,-1,3)$ and $M=\left[\begin{array}{ccc}1& 0& 2\\ 0& -1& 1\end{array}\right]$. The self-dot product is the squared Euclidean length,

$$v^{\top }v=2^2+(-1{)}^2+3^2=14=\Vert v{\Vert }_2^2,$$

the matrix-vector product compresses three coordinates into two,

$$Mv=\left[\begin{array}{c}1(2)+0(-1)+2(3)\\ 0(2)+(-1)(-1)+1(3)\end{array}\right]=\left[\begin{array}{c}8\\ 4\end{array}\right],$$

and the first row of the outer product $v{v}^{\top }$ is $2v=(4,-2,6)$. That same linear-map view underlies linear algebra, dense layers in neural network fundamentals, and low-dimensional embeddings.

Caveats

Shape errors are semantic errors, not just syntax errors: a feature vector stored as shape $(d,)$ can behave differently from a row matrix $(1,d)$ or column matrix $(d,1)$. Units matter too; adding a price feature and an age feature before scaling can make distances meaningless.

References

• MIT OpenCourseWare: 18.06 Linear Algebra

Nav

Section — Mathematical Foundations

← Linear Algebra Matrix Multiplication →