Linear Algebra Summary
Econometrics and Operations Research 2021/2022
,Table of Contents
Week 1..................................................................................................................................................... 3
Eigenvectors and Eigenvalues ............................................................................................................. 3
Diagonalizable Matrices ...................................................................................................................... 3
Difference Equations ........................................................................................................................... 3
Nondiagonalizable Matrices ................................................................................................................ 4
Jordan Normal Forms .......................................................................................................................... 4
Complex Eigenvalues and Eigenvectors .............................................................................................. 5
Week 2..................................................................................................................................................... 6
Markov Processes................................................................................................................................ 6
Symmetric Matrices ............................................................................................................................ 6
Quadratic Forms .................................................................................................................................. 7
Differential Equations.......................................................................................................................... 7
Solutions of First Order ODEs .............................................................................................................. 8
Solutions of Inhomogeneous Linear ODEs .......................................................................................... 8
Week 3..................................................................................................................................................... 9
Second Order Linear ODEs .................................................................................................................. 9
Direction Fields .................................................................................................................................. 10
Phase Portraits .................................................................................................................................. 10
Systems of Differential Equations ..................................................................................................... 11
Linear Systems of ODEs ..................................................................................................................... 12
Week 4................................................................................................................................................... 13
Stability Properties of Equilibrium Solutions..................................................................................... 13
Phase Portraits .................................................................................................................................. 14
Determinants and their Properties ................................................................................................... 15
The Inverse of a Matrix...................................................................................................................... 16
Cramer’s Rule .................................................................................................................................... 17
Week 5................................................................................................................................................... 18
Linear Spaces and Linear Subspaces ................................................................................................. 18
Bases and Dimension of a Linear Space ............................................................................................ 18
Row Spaces ........................................................................................................................................ 19
Column Spaces .................................................................................................................................. 19
Solving Systems of Linear Equations ................................................................................................. 20
Week 6................................................................................................................................................... 21
Null Spaces ........................................................................................................................................ 21
Affine Subspaces ............................................................................................................................... 22
Linear Basis Transformations ............................................................................................................ 22
2
, Week 1
Eigenvectors and Eigenvalues
- Canonical basis vectors in ℝ𝑛 :
1 0 0
0 1 0
- 𝑒1 = ( ) , 𝑒2 = ( ),…, 𝑒1 = ( ).
⋮ ⋮ ⋮
0 0 1
- Identity matrix in 𝑅 𝑛×𝑛 :
1 0 ⋯ 0
0 1 ⋯ 0
- 𝐼𝑛 = ( ).
0 0 ⋱ ⋮
0 0 ⋮ 1
- An eigenvalue of a square matrix 𝐴 is a number 𝜆 ∈ ℂ such that the matrix 𝐴 − 𝜆𝐼𝑛 is
singular.
- The number 𝜆 ∈ ℂ is an eigenvalue of 𝐴 iff det(𝐴 − 𝜆𝐼) = 0. This is the characteristic
equation of matrix 𝐴.
- The trace of a square matrix 𝐴 is the sum of its diagonal entries.
▪ 𝜆1 + 𝜆2 + ⋯ + 𝜆𝑛 = trace(𝐴).
▪ 𝜆1 ∗ 𝜆2 ∗ ⋯ ∗ 𝜆𝑛 = det(𝐴).
- A vector 𝑣 ≠ 0 such that (𝐴 − 𝜆𝐼)𝑣 = 0 for some eigenvalue 𝜆 of 𝐴 is called an eigenvector
of 𝐴 corresponding to 𝜆.
- Note that: (𝐴 − 𝜆𝐼)𝑣 = 0 ⇔ 𝐴𝑣 = 𝜆𝑣.
Diagonalizable Matrices
- Let 𝐴 be a square matrix and let 𝜆1 , … , 𝜆2 , … , 𝜆𝑛 be its eigenvalues and 𝑣1 , 𝑣2 , … , 𝑣𝑛
corresponding eigenvectors, with 𝑃 ≔ [𝑣1 … 𝑣𝑛 ]. Then:
- If P is invertible, then:
𝜆1 0 ⋯ 0
0 𝜆2 ⋯ 0
▪ 𝑃−1 𝐴𝑃 = ( ).
0 0 ⋱ ⋮
0 0 ⋮ 𝜆𝑛
−1
- If 𝑄 𝐴𝑄 is a diagonal matrix 𝐷, then the columns of 𝑄 are eigenvectors of 𝐴 and the
diagonal entries of 𝐷 are eigenvalues of 𝐴.
Difference Equations
- One can use difference equations to model dynamical problems.
- If 𝐴 is a square matrix, then 𝑧𝑡+1 = 𝐴𝑧𝑡 is a system of linear difference equations (LDE).
- If you know the initial value 𝑧0 then you can iteratively calculate 𝑧1 , 𝑧2 , etc.
- In general: 𝑧𝑡 = 𝐴𝑡 𝑧0 , 𝑡 ≥ 0.
- A solution is of an LDE is a sequence of vectors {𝑧𝑡 }∞
𝑡=𝑡0 that ‘fits’ the LDE.
- The general solution of an LDE is the set containing all solutions of the LDE.
- The sequence {0, 0, 0, … } is a solution of an LDE for every 𝐴 and is called the null solution of
the LDE.
3
Econometrics and Operations Research 2021/2022
,Table of Contents
Week 1..................................................................................................................................................... 3
Eigenvectors and Eigenvalues ............................................................................................................. 3
Diagonalizable Matrices ...................................................................................................................... 3
Difference Equations ........................................................................................................................... 3
Nondiagonalizable Matrices ................................................................................................................ 4
Jordan Normal Forms .......................................................................................................................... 4
Complex Eigenvalues and Eigenvectors .............................................................................................. 5
Week 2..................................................................................................................................................... 6
Markov Processes................................................................................................................................ 6
Symmetric Matrices ............................................................................................................................ 6
Quadratic Forms .................................................................................................................................. 7
Differential Equations.......................................................................................................................... 7
Solutions of First Order ODEs .............................................................................................................. 8
Solutions of Inhomogeneous Linear ODEs .......................................................................................... 8
Week 3..................................................................................................................................................... 9
Second Order Linear ODEs .................................................................................................................. 9
Direction Fields .................................................................................................................................. 10
Phase Portraits .................................................................................................................................. 10
Systems of Differential Equations ..................................................................................................... 11
Linear Systems of ODEs ..................................................................................................................... 12
Week 4................................................................................................................................................... 13
Stability Properties of Equilibrium Solutions..................................................................................... 13
Phase Portraits .................................................................................................................................. 14
Determinants and their Properties ................................................................................................... 15
The Inverse of a Matrix...................................................................................................................... 16
Cramer’s Rule .................................................................................................................................... 17
Week 5................................................................................................................................................... 18
Linear Spaces and Linear Subspaces ................................................................................................. 18
Bases and Dimension of a Linear Space ............................................................................................ 18
Row Spaces ........................................................................................................................................ 19
Column Spaces .................................................................................................................................. 19
Solving Systems of Linear Equations ................................................................................................. 20
Week 6................................................................................................................................................... 21
Null Spaces ........................................................................................................................................ 21
Affine Subspaces ............................................................................................................................... 22
Linear Basis Transformations ............................................................................................................ 22
2
, Week 1
Eigenvectors and Eigenvalues
- Canonical basis vectors in ℝ𝑛 :
1 0 0
0 1 0
- 𝑒1 = ( ) , 𝑒2 = ( ),…, 𝑒1 = ( ).
⋮ ⋮ ⋮
0 0 1
- Identity matrix in 𝑅 𝑛×𝑛 :
1 0 ⋯ 0
0 1 ⋯ 0
- 𝐼𝑛 = ( ).
0 0 ⋱ ⋮
0 0 ⋮ 1
- An eigenvalue of a square matrix 𝐴 is a number 𝜆 ∈ ℂ such that the matrix 𝐴 − 𝜆𝐼𝑛 is
singular.
- The number 𝜆 ∈ ℂ is an eigenvalue of 𝐴 iff det(𝐴 − 𝜆𝐼) = 0. This is the characteristic
equation of matrix 𝐴.
- The trace of a square matrix 𝐴 is the sum of its diagonal entries.
▪ 𝜆1 + 𝜆2 + ⋯ + 𝜆𝑛 = trace(𝐴).
▪ 𝜆1 ∗ 𝜆2 ∗ ⋯ ∗ 𝜆𝑛 = det(𝐴).
- A vector 𝑣 ≠ 0 such that (𝐴 − 𝜆𝐼)𝑣 = 0 for some eigenvalue 𝜆 of 𝐴 is called an eigenvector
of 𝐴 corresponding to 𝜆.
- Note that: (𝐴 − 𝜆𝐼)𝑣 = 0 ⇔ 𝐴𝑣 = 𝜆𝑣.
Diagonalizable Matrices
- Let 𝐴 be a square matrix and let 𝜆1 , … , 𝜆2 , … , 𝜆𝑛 be its eigenvalues and 𝑣1 , 𝑣2 , … , 𝑣𝑛
corresponding eigenvectors, with 𝑃 ≔ [𝑣1 … 𝑣𝑛 ]. Then:
- If P is invertible, then:
𝜆1 0 ⋯ 0
0 𝜆2 ⋯ 0
▪ 𝑃−1 𝐴𝑃 = ( ).
0 0 ⋱ ⋮
0 0 ⋮ 𝜆𝑛
−1
- If 𝑄 𝐴𝑄 is a diagonal matrix 𝐷, then the columns of 𝑄 are eigenvectors of 𝐴 and the
diagonal entries of 𝐷 are eigenvalues of 𝐴.
Difference Equations
- One can use difference equations to model dynamical problems.
- If 𝐴 is a square matrix, then 𝑧𝑡+1 = 𝐴𝑧𝑡 is a system of linear difference equations (LDE).
- If you know the initial value 𝑧0 then you can iteratively calculate 𝑧1 , 𝑧2 , etc.
- In general: 𝑧𝑡 = 𝐴𝑡 𝑧0 , 𝑡 ≥ 0.
- A solution is of an LDE is a sequence of vectors {𝑧𝑡 }∞
𝑡=𝑡0 that ‘fits’ the LDE.
- The general solution of an LDE is the set containing all solutions of the LDE.
- The sequence {0, 0, 0, … } is a solution of an LDE for every 𝐴 and is called the null solution of
the LDE.
3
