Matrix Calculus

Terms and definitions

As matrix is called a system of elements a_ij, which are arranged in a 2-dimensional rectangular pattern. The scheme of m-rows and n-columns is called a (m, n)-matrix or a m x n matrix. The position of an element within the matrix is characterized by two subscripts. The first index is the row number and the second index is the column number. The numbering starts at the top left of the matrix and going from left to right and from top to bottom. If for a matrix is n = m then the matrix is called a square matrix.

$A = (a_{i j}) = (\begin{matrix} a_{11} & a_{12} & \dots & a_{1 m} \\ a_{21} & a_{22} & \dots & a_{2 m} \\ ⋮ \\ a_{n 1} & a_{n 2} & \dots & a_{n m} \end{matrix})$

Main Diagonal

The elements of the matrix for the subscripts i = j are the main diagonal elements. The elements from the lower left to upper right are referred as secondary diagonal.

Here the main diagonal elements are shown in red color:

$(\begin{matrix} a_{11} & a_{12} & \dots & a_{1 m} \\ a_{21} & a_{22} & \dots & a_{2 m} \\ ⋮ \\ a_{n 1} & a_{n 2} & \dots & a_{n m} \end{matrix})$

and the secondary diagonal elements in green color:

$(\begin{matrix} a_{11} & a_{12} & \dots & a_{1 m-1} & a_{1 m} \\ a_{21} & a_{22} & \dots & a_{2 m-1} & a_{2 m} \\ ⋮ \\ a_{n 1} & a_{n 2} & \dots & a_{n m-1} & a_{n m} \end{matrix})$

Unit Matrix

The matrix in which all elements of the main diagonal equal to 1 and all other elements are equal to 0 means unit matrix E.

$E = (\begin{matrix} 1 & 0 & \dots & 0 \\ 0 & 1 & \dots & 0 \\ ⋮ \\ 0 & 0 & \dots & 1 \end{matrix})$

Transposed Matrix

The matrix mirrored on the main diagonal is called the matrix transpose. For a matrix A = (a_ij) the transposed matrix A^T = (a_ji). The transposed of a transposed matrix is the matrix itself A = (A^T)^T.

$A^{T} = {(a_{i j})}^{T} = {(\begin{matrix} a_{11} & a_{12} & \dots & a_{1 m} \\ a_{21} & a_{22} & \dots & a_{2 m} \\ ⋮ \\ a_{n 1} & a_{n 2} & \dots & a_{n m} \end{matrix})}^{T} = (\begin{matrix} a_{11} & a_{21} & \dots & a_{n 1} \\ a_{12} & a_{22} & \dots & a_{n 2} \\ ⋮ \\ a_{1 m} & a_{2 m} & \dots & a_{n m} \end{matrix})$

Determinant

Each square matrix can be assigned a unique number, which is called the determinant (det(A)) of the matrix. In general, the determinant of an NxN matrix is defined by the Leibniz formula:

$det A = \sum_{σ \in S_{n}}^{} (sgn (σ) Π_{i = 1}^{n} A_{i ρ (i)})$

here the sum has to be extended over all the permutations σ. Thus, from the elements of A, all possible products are formed for each n-element in such a way that each of the products of each row and column contains exactly one element. These products are added and the sum is the determinant of A. The sign of the summands is positive for even permutations and negative for odd permutations.

Inverse Matrix

The inverse matrix A^-1 is defined by the following equation

$A \cdot A^{-1} = E$

Matrices for which an inverse exists is referred to as regular matrices. Matrices which have no inverse are called singular matrices.

For the inverse matrix, the following calculation rules are valid:

${(A \cdot B)}^{-1} = A^{-1} \cdot B^{-1}$

${(A^{-1})}^{-1} = A$

The calculation of the inverse matrix A^-1 is either done by the Gauss-Jordan algorithm or with the adjuncts. The Gauss-Jordan method transforms the matrix (A | E) in the form (E | A^-1) from which the inverse can be read directly. With the adjuncts and the determinant the inverse can be calculated directly as

$A^{-1} = \frac{1}{\det (A)} {adj(A)}^{T}$

Clases of Matrices

A square matrix A is called a symmetric matrix if and only if A^T = A and a antisymmetric matrix applies if A^T = A. A orthogonal matrix if and only if A^T = A^-1

Adjungate Matrix

The adjunct of matrix A is calculated in a way that for each matrix element a_ij is set a sub determinant with removing the line i and the column j. The value of this determinat is multiplied by (-1)^i+j that gives the element i,j of the adjungate matrix.

Calculation Rules for Matrices

The matrix multiplication is associative:

$A \cdot (B \cdot C) = (A \cdot B) \cdot C$

The matrix multiplication and matrix addition are distributive:

$A \cdot (B + C) = A \cdot B + A \cdot C$

For addition and multiplication by real numbers λ, μ:

$(λ + μ) \cdot A = λ \cdot A + μ \cdot A$

and:

$λ \cdot (A + B) = λ \cdot A + λ \cdot B$

There are zero divisor matrices A ≠ 0 and B ≠ 0 applies to

$A \cdot B = 0$

For square matrices is:

$\det (A + B) = \det (A) + \det (B)$

Matrix Summation

The addition of two matrices A and B is done by adding the elements of the matrices. C = A + B with c_{i, j} = a_{i, j} + b _{i, j}

$(\begin{matrix} a_{11} & a_{12} & \dots & a_{1 m} \\ a_{21} & a_{22} & \dots & a_{2 m} \\ ⋮ \\ a_{n 1} & a_{n 2} & \dots & a_{n m} \end{matrix}) + (\begin{matrix} b_{11} & b_{12} & \dots & b_{1 m} \\ b_{21} & b_{22} & \dots & b_{2 m} \\ ⋮ \\ b_{n 1} & b_{n 2} & \dots & b_{n m} \end{matrix}) = (\begin{matrix} a_{11} + b_{11} & a_{12} + b_{12} & \dots & a_{1 m} + b_{1 m} \\ a_{21} + b_{21} & a_{22} + b_{22} & \dots & a_{2 m} + b_{2 m} \\ ⋮ \\ a_{n 1} + b_{n 1} & a_{n 2} + b_{n 2} & \dots & a_{n m} + b_{n m} \end{matrix})$

Calculator for the addition of two matrices:

+

=

A general calculator for the sum of NxM matrices is here: Sum and dif of MxN matrices

Matrices Subtraction

The subtraction of two matrices A and B is by subtracting the elements of the matrices. C = A - B with c _{i, j} = a _{i, j} - b _{i, j}

$(\begin{matrix} a_{11} & a_{12} & \dots & a_{1 m} \\ a_{21} & a_{22} & \dots & a_{2 m} \\ ⋮ \\ a_{n 1} & a_{n 2} & \dots & a_{n m} \end{matrix}) - (\begin{matrix} b_{11} & b_{12} & \dots & b_{1 m} \\ b_{21} & b_{22} & \dots & b_{2 m} \\ ⋮ \\ b_{n 1} & b_{n 2} & \dots & b_{n m} \end{matrix}) = (\begin{matrix} a_{11} - b_{11} & a_{12} - b_{12} & \dots & a_{1 m} - b_{1 m} \\ a_{21} - b_{21} & a_{22} - b_{22} & \dots & a_{2 m} - b_{2 m} \\ ⋮ \\ a_{n 1} - b_{n 1} & a_{n 2} - b_{n 2} & \dots & a_{n m} - b_{n m} \end{matrix})$

Calculator for subtraction of two matrices:

-

=

A general calculator for the subtraction of NxM matrices is here: Sum and dif of MxN matrices

Matrix multiplication by a scalar

Multiplying a matrix by a scalar is by multiplying each by the scalar matrix elements. a * B = a * b_i,j

$λ \cdot (\begin{matrix} a_{11} & a_{12} & \dots & a_{1 m} \\ a_{21} & a_{22} & \dots & a_{2 m} \\ ⋮ \\ a_{n 1} & a_{n 2} & \dots & a_{n m} \end{matrix}) = (\begin{matrix} λ \cdot a_{11} & λ \cdot a_{12} & \dots & λ \cdot a_{1 m} \\ λ \cdot a_{21} & λ \cdot a_{22} & \dots & λ \cdot a_{2 m} \\ ⋮ \\ λ \cdot a_{n 1} & λ \cdot a_{n 2} & \dots & λ \cdot a_{n m} \end{matrix})$

Matrix Multiplication

The multiplication of two matrices A and B requires that the number of columns of the first matrix is equal to the number of rows of the second matrix. The product obtained by multiplying the row and column elements and summed. For the first element of the result matrix, the elements of the first row of the first matrix are multiplied by the elements of the first column of the second matrix and summed. For the other elements the same for the other rows and columns.

$(\begin{matrix} a_{11} & a_{12} & \dots & a_{1 m} \\ a_{21} & a_{22} & \dots & a_{2 m} \\ ⋮ \\ a_{n 1} & a_{n 2} & \dots & a_{n m} \end{matrix}) \cdot (\begin{matrix} b_{11} & b_{12} & \dots & b_{1 j} \\ b_{21} & b_{22} & \dots & b_{2 j} \\ ⋮ \\ b_{m 1} & b_{m 2} & \dots & b_{m j} \end{matrix}) = (\begin{matrix} \sum_{k = 1}^{m} (a_{1 k} \cdot b_{k 1}) & \sum_{k = 1}^{m} (a_{1 k} \cdot b_{k 2}) & \dots & \sum_{k = 1}^{m} (a_{1 k} \cdot b_{k j}) \\ \sum_{k = 1}^{m} (a_{2 k} \cdot b_{k 1}) & \sum_{k = 1}^{m} (a_{2 k} \cdot b_{k 2}) & \dots & \sum_{k = 1}^{m} (a_{2 k} \cdot b_{k j}) \\ ⋮ \\ \sum_{k = 1}^{m} (a_{n k} \cdot b_{k 1}) & \sum_{k = 1}^{m} (a_{n k} \cdot b_{k 2}) & \dots & \sum_{k = 1}^{m} (a_{n k} \cdot b_{k j}) \end{matrix})$

A general calculator for the multiplication of NxM matrices is here: Multiplication of matrices

Sarrus rule

The determinant of a square 3x3 matrix is computed according to the Sarrus rule by subtracting the sum of the products of the main diagonal of the sum of the products of the secondary diagonal.

A general determinant solver is here:Determinant NxN

Calculation of the Inverse by Gauss-Jordan

Wanted is the inverse matrix A^-1 to the matrix A. For this, first with the identity matrix, the matrix E (A | E) is formed. By suitable transformations we managed to form the (E | A ^-1). In the following the steps of an example can be performed.

$A = (\begin{matrix} a_{11} & a_{12} & \dots & a_{1 N} \\ a_{21} & a_{22} & \dots & a_{2 N} \\ ⋮ \\ a_{N 1} & a_{N 2} & \dots & a_{N N} \end{matrix})$

Gauss-Jordan approach

$(A | E) = (\begin{matrix} a_{11} & a_{12} & \dots & a_{1 N} \\ a_{21} & a_{22} & \dots & a_{2 N} \\ ⋮ \\ a_{N 1} & a_{N 2} & \dots & a_{N N} \end{matrix} | \begin{matrix} 1 & 0 & \dots & 0 \\ 0 & 1 & \dots & 0 \\ ⋮ \\ 0 & 0 & \dots & 1 \end{matrix})$

Transformations to get the following shape.

$(E | A^{-1}) =$ $(\begin{matrix} 1 & 0 & \dots & 0 \\ 0 & 1 & \dots & 0 \\ ⋮ \\ 0 & 0 & \dots & 1 \end{matrix} | \begin{matrix} b_{11} & b_{12} & \dots & b_{1 N} \\ b_{21} & b_{22} & \dots & b_{2 N} \\ ⋮ \\ b_{N 1} & b_{N 2} & \dots & b_{N N} \end{matrix})$

Calculator for the inverse matrix: Solver Inverse Matrix

Calculation of the adjugate matrix

The adjunct of matrix A is calculated in a way that for each matrix element a_ij is set a sub determinant with removing the line i and the column j. The value of this determinat is multiplied by (-1)^i+j that gives the element i,j of the adjungate matrix.

$a_{i j}^{*} = {(- 1)}^{(i + j)} | \begin{matrix} a_{11} & a_{12} & \dots & a_{1, j - 1} & a_{1, j + 1} & \dots & a_{1 n} \\ ⋮ \\ a_{i - 1, 1} & a_{i - 1, 2} & \dots & a_{i - 1, j - 1} & a_{i - 1, j + 1} & \dots & a_{i - 1, n} \\ a_{i + 1, 1} & a_{i + 1, 2} & \dots & a_{i + 1, j - 1} & a_{i + 1, j + 1} & \dots & a_{i + 1, n} \\ ⋮ \\ a_{n 1} & a_{n 2} & \dots & a_{n, j - 1} & a_{n, j + 1} & \dots & a_{n n} \end{matrix} |$

The result is the adjungate matrix.

$adj(A)= (\begin{matrix} a_{11}^{*} & a_{12}^{*} & \dots & a_{1 n}^{*} \\ a_{21}^{*} & a_{22}^{*} & \dots & a_{2 n}^{*} \\ ⋮ \\ a_{n 1}^{*} & a_{n 1}^{*} & \dots & a_{n n}^{*} \end{matrix})$

Calculator for the adjugate matrix: Solver Adjugate matrix

Multiplication of a vector with a matrix

The product of a matrix with a vector is a linear image. The multiplication is explained if the number of columns of the matrix is equal to the number of elements of the vector. The result is a vector whose number of components equals the number of rows of the matrix. This means that a matrix with 2 rows always maps a vector to a vector with two components.

$A \cdot \vec{v} = (\begin{matrix} a_{11} & a_{12} & \dots & a_{1 m} \\ a_{21} & a_{22} & \dots & a_{2 m} \\ ⋮ \\ a_{n 1} & a_{n 2} & \dots & a_{n m} \end{matrix}) \cdot (\begin{matrix} v_{1} \\ v_{2} \\ ⋮ \\ v_{m} \end{matrix}) = (\begin{matrix} a_{11} v_{1} + a_{12} v_{2} + \dots + a_{1 m} v_{m} \\ a_{21} v_{1} + a_{22} v_{2} + \dots + a_{2 m} v_{m} \\ ⋮ \\ a_{n 1} v_{1} + a_{n 2} v_{2} + \dots + a_{n m} v_{m} \end{matrix})$

Calculator for the matrix-vector product: Graphical Matrix-Vector product

Eigenvalue Calculation

The equation

$A v = λ v$

can be transformed into the homogeneous equation system

$(A - λ E) v = 0$

The system of equations has a non-trivial solution if and only if the determinant disappears. That if applicable

$\det (A - λ E) = 0$

The polynomial is called the characteristic polynomial of A and the equation is the characteristic equation of A. If λ_i is an eigenvalue of A then the solutions of the characteristic equation are the eigenvectors of A to the eigenvalue λ_i.

Calculator for eigenvalues: Eigenvalues