Covariance Matrix — Formula, Properties, Examples

TL;DR

A covariance matrix is a square matrix that holds the variance of each variable on its diagonal and the covariance between each pair of variables off the diagonal — and it is always symmetric and positive semi-definite. This article covers the definition, the variance and covariance formulas, the key properties, a full 2×2 worked example, and how the matrix powers principal component analysis.

What Is a Covariance Matrix?

A covariance matrix is a square matrix that summarises how a set of variables vary — individually and together. For k variables, it is a k×k matrix where the diagonal entries are the variances of each variable and the off-diagonal entries are the covariances between each pair. It is also called the variance–covariance matrix.

Covariance itself measures whether two variables move together: positive when they rise and fall together, negative when one rises as the other falls, and zero when they have no linear relationship. The covariance matrix collects all of these into one object, generalising the single-variable idea of variance to many dimensions at once. Because each entry is built from data, the whole matrix is a matrix you compute rather than one you are simply handed.

What Is the Covariance Matrix Formula?

Two formulas build the matrix: one for the diagonal (variance) and one for the off-diagonal (covariance). For a sample of n data points, the sample variance of variable X with mean (\bar{X}) is:

[ \text{Var}(X) = \frac{1}{n-1}\sum_{i=1}^{n}(X_{i} - \bar{X})^2, ]

and the sample covariance between X and Y is:

[ \text{Cov}(X, Y) = \frac{1}{n-1}\sum_{i=1}^{n}(X_{i} - \bar{X})(Y_{i} - \bar{Y}). ]

The full matrix for two variables is then assembled by placing variances on the diagonal and the shared covariance in both off-diagonal slots:

[ \Sigma = \begin{bmatrix} \text{Var}(X) & \text{Cov}(X, Y) \\n\text{Cov}(X, Y) & \text{Var}(Y) \end{bmatrix}. ]

The off-diagonal entries are identical because (\text{Cov}(X, Y) = \text{Cov}(Y, X)) — which is why every covariance matrix is symmetric.

What does the covariance matrix look like for more than two variables?

It grows to match. Three variables give a 3×3 matrix, with the three variances down the diagonal and the three distinct pairwise covariances (each appearing twice, mirrored across the diagonal) filling the rest:

[ \Sigma = \begin{bmatrix} \text{Var}(X) & \text{Cov}(X,Y) & \text{Cov}(X,Z) \\n\text{Cov}(X,Y) & \text{Var}(Y) & \text{Cov}(Y,Z) \\n\text{Cov}(X,Z) & \text{Cov}(Y,Z) & \text{Var}(Z) \end{bmatrix}. ]

The pattern holds for any number of variables: a k×k symmetric matrix.

What Are the Properties of a Covariance Matrix?

Every covariance matrix obeys a short, reliable list of rules. Each one is worth knowing because together they constrain what a valid covariance matrix can even look like.

• Square. For k variables it is always k×k.
• Symmetric. (\Sigma^{T} = \Sigma), because (\text{Cov}(X,Y) = \text{Cov}(Y,X)).
• Positive semi-definite. For any vector v, (v^{T}\Sigma v \geq 0). Intuitively, no combination of variables can have negative variance.
• Diagonal entries are non-negative. Each is a variance, and a variance is never negative.
• Real, non-negative eigenvalues. A direct consequence of being symmetric positive semi-definite.

Examples of Covariance Matrix

Example 1

Find (\text{Cov}(X,Y)) for X={2, 4, 6} and Y={1, 3, 5}.
Means: (\bar{X} = 4), (\bar{Y} = 3). Deviations multiplied and summed:

[ (2-4)(1-3)+(4-4)(3-3)+(6-4)(5-3) = 8. ]

Divide by n−1=2:

[\text{Cov}(X,Y) = \frac{8}{2} = 4.]

Final answer: (\text{Cov}(X, Y) = 4).

Example 2

A common slip — recompute the sample covariance above using the right divisor.
Wrong attempt. A student computes the sum of deviation products correctly as 8, then divides by n=3 to get (\text{Cov}(X, Y) = \frac{8}{3} \approx 2.67).
Correct. For a sample covariance, the divisor is n−1, not n.

Example 3

Build the 2×2 covariance matrix for X={2, 4, 6} and Y={1, 3, 5}.
From Example 1, Cov(X,Y)=4. Now the variances.

[\text{Var}(X) = \frac{(2-4)^2 + 0 + (6-4)^2}{2} = 4,]

[\text{Var}(Y) = \frac{(1-3)^2 + 0 + (5-3)^2}{2} = 4.]

Assemble:

[\Sigma = \begin{bmatrix} 4 & 4 \\n4 & 4 \end{bmatrix}. ]

Example 4

For X={10, 5} and Y={3, 9}, the covariance matrix is reported as:
[\begin{bmatrix} 6.25 & -15 \\n-15 & 9 \end{bmatrix}].
The off-diagonal entry is (\text{Cov}(X, Y) = -15), a negative number.
Final answer: The negative covariance means X and Y move in opposite directions.

Example 5

Is (M = \begin{bmatrix} 5 & 2 \\n3 & 8 \end{bmatrix}) a valid covariance matrix?
Here 2≠3.
Final answer: No. Because M is not symmetric, it cannot be a covariance matrix.

Example 6

A covariance matrix has (\text{Cov}(X,Y) = 0).
Final answer: X and Y are uncorrelated.

Why the Covariance Matrix Runs Modern Data Science

The covariance matrix answers exactly which direction data spreads — driving techniques in computing and finance.

• Principal component analysis (PCA). The eigenvectors point along the directions of greatest variance in the data.
• Portfolio risk in finance. The covariance matrix of asset returns is the core input to portfolio theory.
• The multivariate normal distribution. The covariance matrix sets the shape of the probability cloud under statistical models.

Where Students Trip Up on the Covariance Matrix

Mistake 1: Using the wrong divisor (n vs n − 1)

Mistake 2: Confusing covariance with correlation

Mistake 3: Forgetting the matrix must be symmetric

Key Takeaways

• A covariance matrix is a square matrix with variances on the diagonal and covariances off the diagonal.
• It is always symmetric and positive semi-definite, with real non-negative eigenvalues.
• Sample variance and covariance divide by n−1; population divides by n.
• Covariance is unbounded and scale-dependent; correlation is not.