Vectors, Matrices, and Limits
Scalars and Vectors
\mathbf{v} = \begin{bmatrix} v_1 \\ v_2 \\ v_3 \end{bmatrix} \quad \text{e.g.} \quad \mathbf{v} = \begin{bmatrix} 2 \\ -1 \\ 4 \end{bmatrix}
A vector is just coordinates: an ordered list of n numbers places one point in n-dimensional space
Multiply every entry by the scalar — the vector stretches, but keeps its direction
c\mathbf{v} = \begin{bmatrix} cv_1 & cv_2 & \dots & cv_n \end{bmatrix}
Let c = 3 and \mathbf{v} = \begin{bmatrix} 2 \\ -1 \\ 4 \end{bmatrix}
c\mathbf{v} = 3 \cdot \begin{bmatrix} 2 \\ -1 \\ 4 \end{bmatrix} = \begin{bmatrix} 6 \\ -3 \\ 12 \end{bmatrix}
Add or subtract entry by entry — vectors must have the same length
\mathbf{U} = \begin{bmatrix} 2 \\ 4 \\ 6 \end{bmatrix}; \quad \mathbf{V} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}
\mathbf{U} - \mathbf{V} = \begin{bmatrix} 2-1 \\ 4-2 \\ 6-3 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}
Multiply corresponding entries and sum: the output is a single scalar, not a vector
\mathbf{u} \cdot \mathbf{v} = u_1v_1 + u_2v_2 + \dots + u_nv_n = \sum_{i=1}^{n} u_i v_i
Example: Let \mathbf{u} = \begin{bmatrix} 1 \\ 3 \\ -5 \end{bmatrix}, \mathbf{v} = \begin{bmatrix} 4 \\ -2 \\ -1 \end{bmatrix}
\mathbf{u} \cdot \mathbf{v} = (1)(4) + (3)(-2) + (-5)(-1) = 4 - 6 + 5 = 3
Take-home: (1) learn the mechanical steps above; (2) the intuition is a scaled, weighted shadow: how far \mathbf{v} falls along \mathbf{u}, scaled by \mathbf{u}’s length
The straight-line distance between two points \mathbf{p} = (p_1, p_2) and \mathbf{q} = (q_1, q_2)
d(\mathbf{p}, \mathbf{q}) = \sqrt{(q_1 - p_1)^2 + (q_2 - p_2)^2}
Why Pythagoras applies: mark the corner point \mathbf{r} = (q_1, p_1). Going from \mathbf{p} to \mathbf{r} only changes the x-coordinate (horizontal, length |q_1-p_1|); going from \mathbf{r} to \mathbf{q} only changes the y-coordinate (vertical, length |q_2-p_2|) — and a horizontal segment always meets a vertical one at a right angle. So \mathbf{p}, \mathbf{r}, \mathbf{q} form a right triangle, and Pythagoras hands us the hypotenuse, d(\mathbf{p}, \mathbf{q}), directly
Intuition: e.g. \mathbf{p} = a voter’s position, \mathbf{q} = a candidate’s position — this formula tells us how far apart they are, and the smaller the distance, the closer their views (more on this with Downs shortly)
Example: Let \mathbf{P} = (1, 2) and \mathbf{Q} = (4, 6)
d(\mathbf{P}, \mathbf{Q}) = \sqrt{(4 - 1)^2 + (6 - 2)^2} = \sqrt{3^2 + 4^2} = \sqrt{9 + 16} = \sqrt{25} = 5
The Euclidean distance between \mathbf{P} and \mathbf{Q} is 5 units
Downs, A. (1957). An Economic Theory of Democracy. Harper & Row.
Voters and candidates as points in a policy space:
Closer candidate \Rightarrow higher voter utility
Two parties negotiating a coalition each bring an issue-priority vector (Healthcare, Tax, Defence): \mathbf{a} = \begin{pmatrix} 2 \\ 3 \\ -1 \end{pmatrix}, \mathbf{b} = \begin{pmatrix} -1 \\ 4 \\ 2 \end{pmatrix}. Find the coalition’s combined platform, \mathbf{a} + \mathbf{b}.
\mathbf{a} + \mathbf{b} = \begin{pmatrix} 2 + (-1) \\ 3 + 4 \\ -1 + 2 \end{pmatrix} = \begin{pmatrix} 1 \\ 7 \\ 1 \end{pmatrix}
Practice Time!
Matrices
An array of numbers, m rows by n columns — a vector is just a one-row or one-column matrix
\mathbf{U} = \begin{bmatrix} u_{11} & u_{12} & u_{13} \\ u_{21} & u_{22} & u_{23} \\ u_{31} & u_{32} & u_{33} \end{bmatrix} \quad \text{e.g.} \quad \mathbf{U} = \begin{bmatrix} 2 & 1 & 0 \\ 4 & 2 & 1 \\ 6 & 3 & 2 \end{bmatrix}
Matrices \mathbf{A} and \mathbf{B} must have the same dimensionality
\mathbf{A} = \begin{bmatrix} 2 & 1 & 0 \\ 4 & 2 & 1 \\ 6 & 3 & 2 \end{bmatrix}, \quad \mathbf{B} = \begin{bmatrix} 2 & 1 & 1 \\ 1 & 3 & 4 \\ 5 & 1 & 2 \end{bmatrix}
\mathbf{A} + \mathbf{B} = \begin{bmatrix} 2+2 & 1+1 & 0+1 \\ 4+1 & 2+3 & 1+4 \\ 6+5 & 3+1 & 2+2 \end{bmatrix} = \begin{bmatrix} 4 & 2 & 1 \\ 5 & 5 & 5 \\ 11 & 4 & 4 \end{bmatrix}
Matrices \mathbf{A} and \mathbf{B} must have the same dimensionality
\mathbf{A} = \begin{bmatrix} 2 & 5 & 0 \\ 0 & 1 & 2 \end{bmatrix}, \quad \mathbf{B} = \begin{bmatrix} 7 & 1 & 3 \\ 5 & 1 & 2 \end{bmatrix}
\mathbf{A} - \mathbf{B} = \begin{bmatrix} 2-7 & 5-1 & 0-3 \\ 0-5 & 1-1 & 2-2 \end{bmatrix} = \begin{bmatrix} -5 & 4 & -3 \\ -5 & 0 & 0 \end{bmatrix}
Scalar \times matrix multiplication is always possible, regardless of the matrix’s dimensions
\mathbf{A} = \begin{bmatrix} 2 & 1 & 0 \\ 4 & 2 & 1 \\ 6 & 3 & 2 \end{bmatrix}, \quad s = 5
s\mathbf{A} = \begin{bmatrix} 2 \times 5 & 1 \times 5 & 0 \times 5 \\ 4 \times 5 & 2 \times 5 & 1 \times 5 \\ 6 \times 5 & 3 \times 5 & 2 \times 5 \end{bmatrix} = \begin{bmatrix} 10 & 5 & 0 \\ 20 & 10 & 5 \\ 30 & 15 & 10 \end{bmatrix}
\mathbf{A} (m\times k) times \mathbf{B} (k\times n) gives \mathbf{C}=\mathbf{AB} (m \times n) — inner dimensions must match
General rule: entry (i,j) of \mathbf{C} is row i of \mathbf{A} dotted with column j of \mathbf{B}: c_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \cdots + a_{ik}b_{kj} = \sum_{r=1}^{k} a_{ir}b_{rj}
e.g. c_{11} = a_{11}b_{11} + a_{12}b_{21} (row 1 of \mathbf{A} \cdot column 1 of \mathbf{B})
c_{12} = a_{11}b_{12} + a_{12}b_{22} (row 1 of \mathbf{A} \cdot column 2 of \mathbf{B})
…and so on for every (i,j) — the numeric example on the next slide follows this recipe exactly
Using the general rule from the previous slide:
\mathbf{A} = \begin{bmatrix} 2 & 1 \\ 4 & 2 \\ 6 & 3 \end{bmatrix}, \quad \mathbf{B} = \begin{bmatrix} 2 & 1 \\ 1 & 3 \end{bmatrix}
\mathbf{AB} = \begin{bmatrix} 2(2){+}1(1) & 2(1){+}1(3) \\ 4(2){+}2(1) & 4(1){+}2(3) \\ 6(2){+}3(1) & 6(1){+}3(3) \end{bmatrix} = \begin{bmatrix} 5 & 5 \\ 10 & 10 \\ 15 & 15 \end{bmatrix}
\mathbf{C} = \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 1 \\ 1 & 1 & 0 \end{bmatrix}, \quad \mathbf{D} = \begin{bmatrix} 2 & 1 \\ 3 & 4 \\ 1 & 2 \end{bmatrix}
\mathbf{CD} = \begin{bmatrix} 1(2){+}0(3){+}2(1) & 1(1){+}0(4){+}2(2) \\ 0(2){+}1(3){+}1(1) & 0(1){+}1(4){+}1(2) \\ 1(2){+}1(3){+}0(1) & 1(1){+}1(4){+}0(2) \end{bmatrix} = \begin{bmatrix} 4 & 5 \\ 4 & 6 \\ 5 & 5 \end{bmatrix}
Note: matrix division does not exist
Addition: (\mathbf{A} + \mathbf{B}) + \mathbf{C} = \mathbf{A} + (\mathbf{B} + \mathbf{C})
\mathbf{A} = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}, \quad \mathbf{B} = \begin{bmatrix} 2 & 0 \\ 1 & 3 \end{bmatrix}, \quad \mathbf{C} = \begin{bmatrix} 0 & 1 \\ 4 & 2 \end{bmatrix}
(\mathbf{A} + \mathbf{B}) + \mathbf{C} = \begin{bmatrix} 3 & 2 \\ 4 & 7 \end{bmatrix} + \begin{bmatrix} 0 & 1 \\ 4 & 2 \end{bmatrix} = \begin{bmatrix} 3 & 3 \\ 8 & 9 \end{bmatrix}
\mathbf{A} + (\mathbf{B} + \mathbf{C}) = \mathbf{A} + \begin{bmatrix} 2 & 1 \\ 5 & 5 \end{bmatrix} = \begin{bmatrix} 3 & 3 \\ 8 & 9 \end{bmatrix}
Multiplication: (\mathbf{AB})\mathbf{C} = \mathbf{A}(\mathbf{BC})
\mathbf{A} = \begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix}, \quad \mathbf{B} = \begin{bmatrix} 3 & 0 \\ 1 & 4 \end{bmatrix}, \quad \mathbf{C} = \begin{bmatrix} 2 & 1 \\ 1 & 3 \end{bmatrix}
\mathbf{AB} = \begin{bmatrix} 5 & 8 \\ 1 & 4 \end{bmatrix}, \quad \mathbf{BC} = \begin{bmatrix} 6 & 3 \\ 6 & 13 \end{bmatrix}
(\mathbf{AB})\mathbf{C} = \begin{bmatrix} 5 & 8 \\ 1 & 4 \end{bmatrix}\begin{bmatrix} 2 & 1 \\ 1 & 3 \end{bmatrix} = \begin{bmatrix} 18 & 29 \\ 6 & 13 \end{bmatrix}
\mathbf{A}(\mathbf{BC}) = \begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix}\begin{bmatrix} 6 & 3 \\ 6 & 13 \end{bmatrix} = \begin{bmatrix} 18 & 29 \\ 6 & 13 \end{bmatrix}
Subtraction: (\mathbf{A} - \mathbf{B}) - \mathbf{C} \neq \mathbf{A} - (\mathbf{B} - \mathbf{C})
\mathbf{A} = \begin{bmatrix} 4 & 5 \\ 6 & 7 \end{bmatrix}, \quad \mathbf{B} = \begin{bmatrix} 2 & 3 \\ 1 & 4 \end{bmatrix}, \quad \mathbf{C} = \begin{bmatrix} 0 & 1 \\ 2 & 2 \end{bmatrix}
(\mathbf{A} - \mathbf{B}) - \mathbf{C} = \begin{bmatrix} 2 & 2 \\ 5 & 3 \end{bmatrix} - \begin{bmatrix} 0 & 1 \\ 2 & 2 \end{bmatrix} = \begin{bmatrix} 2 & 1 \\ 3 & 1 \end{bmatrix}
\mathbf{A} - (\mathbf{B} - \mathbf{C}) = \mathbf{A} - \begin{bmatrix} 2 & 2 \\ -1 & 2 \end{bmatrix} = \begin{bmatrix} 2 & 3 \\ 7 & 5 \end{bmatrix}
Fun fact: even though regrouping changes the answer, distributing the minus sign does not: \mathbf{A} - (\mathbf{B} - \mathbf{C}) = \mathbf{A} - \mathbf{B} + \mathbf{C} always holds — check: \mathbf{A}-\mathbf{B}+\mathbf{C} = \begin{bmatrix} 2 & 2 \\ 5 & 3 \end{bmatrix} + \begin{bmatrix} 0 & 1 \\ 2 & 2 \end{bmatrix} = \begin{bmatrix} 2 & 3 \\ 7 & 5 \end{bmatrix}, matching \mathbf{A}-(\mathbf{B}-\mathbf{C}) above. This is not the same claim as associativity — it’s just carefully distributing a minus sign
Addition: \mathbf{A} + \mathbf{B} = \mathbf{B} + \mathbf{A} Subtraction: \mathbf{A} - \mathbf{B} \neq \mathbf{B} - \mathbf{A}
\mathbf{A} = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}, \quad \mathbf{B} = \begin{bmatrix} 2 & 0 \\ 1 & 3 \end{bmatrix}
\mathbf{A} + \mathbf{B} = \begin{bmatrix} 3 & 2 \\ 4 & 7 \end{bmatrix} = \mathbf{B} + \mathbf{A}
\mathbf{A} - \mathbf{B} = \begin{bmatrix} -1 & 2 \\ 2 & 1 \end{bmatrix} \neq \mathbf{B} - \mathbf{A} = \begin{bmatrix} 1 & -2 \\ -2 & -1 \end{bmatrix}
Multiplication: \mathbf{AB} \neq \mathbf{BA}, in general
\mathbf{A} = \begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix}, \quad \mathbf{B} = \begin{bmatrix} 3 & 0 \\ 1 & 4 \end{bmatrix}
\mathbf{AB} = \begin{bmatrix} 5 & 8 \\ 1 & 4 \end{bmatrix}, \quad \mathbf{BA} = \begin{bmatrix} 3 & 6 \\ 1 & 6 \end{bmatrix}
\mathbf{AB} \neq \mathbf{BA}: matrix multiplication is not commutative
Left Distributive: \mathbf{A}(\mathbf{B} + \mathbf{C}) = \mathbf{AB} + \mathbf{AC}
\mathbf{A} = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}, \quad \mathbf{B} = \begin{bmatrix} 2 & 0 \\ 1 & 3 \end{bmatrix}, \quad \mathbf{C} = \begin{bmatrix} 0 & 1 \\ 4 & 2 \end{bmatrix}
\mathbf{A}(\mathbf{B} + \mathbf{C}) = \mathbf{A} \begin{bmatrix} 2 & 1 \\ 5 & 5 \end{bmatrix} = \begin{bmatrix} 12 & 11 \\ 26 & 23 \end{bmatrix}
\mathbf{AB} + \mathbf{AC} = \begin{bmatrix} 4 & 6 \\ 10 & 12 \end{bmatrix} + \begin{bmatrix} 8 & 5 \\ 16 & 11 \end{bmatrix} = \begin{bmatrix} 12 & 11 \\ 26 & 23 \end{bmatrix}
Right Distributive: (\mathbf{A} + \mathbf{B})\mathbf{C} = \mathbf{AC} + \mathbf{BC}
(using the same \mathbf{A}, \mathbf{B}, \mathbf{C} as the previous slide)
(\mathbf{A} + \mathbf{B})\mathbf{C} = \begin{bmatrix} 3 & 2 \\ 4 & 7 \end{bmatrix}\begin{bmatrix} 0 & 1 \\ 4 & 2 \end{bmatrix} = \begin{bmatrix} 8 & 7 \\ 28 & 18 \end{bmatrix}
\mathbf{AC} + \mathbf{BC} = \begin{bmatrix} 8 & 5 \\ 16 & 11 \end{bmatrix} + \begin{bmatrix} 0 & 2 \\ 12 & 7 \end{bmatrix} = \begin{bmatrix} 8 & 7 \\ 28 & 18 \end{bmatrix}
Markov, A. A. (1906). Extension of the limit theorems of probability theory to a sum of variables connected in a chain.
Matrices can model voter transitions between parties, election to election. \mathbf{P}: transition probabilities
\mathbf{P} = \begin{bmatrix} 0.8 & 0.1 & 0.1 \\ 0.2 & 0.6 & 0.2 \\ 0.3 & 0.3 & 0.4 \end{bmatrix}
\mathbf{v} = \begin{bmatrix} 0.5 & 0.3 & 0.2 \end{bmatrix}: today’s shares, A, B, C in order
Next period’s distribution: \mathbf{v}' = \mathbf{v}\mathbf{P}
\mathbf{v}' = \mathbf{v}\mathbf{P} = \begin{bmatrix} 0.52 & 0.29 & 0.19 \end{bmatrix}
Next election: 52% Party A, 29% Party B, 19% Party C
Each entry of \mathbf{v}' = \mathbf{v}\mathbf{P} is \mathbf{v} dotted with the matching column of \mathbf{P} — same rule as before, just with a row vector standing in for a 1\times3 matrix
Party A (column 1 of \mathbf{P}): \;0.5(0.8) + 0.3(0.2) + 0.2(0.3) = 0.4+0.06+0.06 = 0.52
Party B (column 2 of \mathbf{P}): \;0.5(0.1) + 0.3(0.6) + 0.2(0.3) = 0.05+0.18+0.06 = 0.29
Party C (column 3 of \mathbf{P}): \;0.5(0.1) + 0.3(0.2) + 0.2(0.4) = 0.05+0.06+0.08 = 0.19
\mathbf{v}' = \begin{bmatrix} 0.52 & 0.29 & 0.19 \end{bmatrix}, \qquad 0.52+0.29+0.19 = 1 \; \checkmark
Practice Time!
Limits
Two preliminary notions:
Jacobson, G. C. (1978). The effects of campaign spending in congressional elections.
Campaigns show diminishing returns: more spending helps, but by less and less. Voter support function:
f(x) = \frac{100x}{x+1}
As spending x \to \infty:
\lim_{x \to \infty} \frac{100x}{x+1} = 100
\lim_{x \to c} f(x) = L
Let’s work it out for \frac{x^2-1}{x-1} as x \to 1:
\frac{x^2 - 1}{x - 1} \bigg|_{x=1} = \frac{1^2 - 1}{1 - 1} = \frac{0}{0}
(indeterminate)
Instead, approach x=1 closer and closer:
| x | 0.50 | 0.90 | 0.99 | 0.999 |
|---|---|---|---|---|
| \frac{x^2-1}{x-1} | 1.50 | 1.90 | 1.990 | 1.9990 |
As x gets closer to 1, the expression gets closer to 2:
\lim_{x \to 1} \frac{x^2 - 1}{x - 1} = 2
| x | 1 | 2 | 10 | 1000 |
|---|---|---|---|---|
| \frac{1}{x} | 1.0 | 0.50 | 0.10 | 0.001 |
As x gets larger, \frac{1}{x} tends towards 0:
\lim_{x \to \infty} \frac{1}{x} = 0
Note: \frac{1}{x} is undefined at x=0 — simply not in its domain — but it’s still perfectly continuous everywhere it is defined, i.e. at every x \neq 0. A gap in the domain is not the same thing as a discontinuity like the jump we saw earlier
Types
Properties
\lim_{x \to c} [f{+}g] = \lim_{x \to c} f + \lim_{x \to c} g
\lim_{x \to c} [\alpha f] = \alpha \lim_{x \to c} f
\lim_{x \to c} [fg] = \left[\lim_{x \to c} f\right] \left[\lim_{x \to c} g\right]
\lim_{x \to c} \left[\frac{f}{g}\right] = \frac{\lim_{x \to c} f}{\lim_{x \to c} g} \; \left(\lim_{x \to c} g \neq 0\right)
Left-hand limit \displaystyle\lim_{x \to c^-} f(x): the value f(x) approaches as x approaches c from below (smaller values). Right-hand limit \displaystyle\lim_{x \to c^+} f(x): the value f(x) approaches as x approaches c from above (larger values)
Why this matters: the two-sided limit \lim_{x \to c} f(x) exists only if the left- and right-hand limits both exist and are equal. This is essential for a piecewise function, whose formula changes at c
Worked example: let \displaystyle g(x) = \begin{cases} x + 1 & x < 2 \\ x^2 - 1 & x \geq 2 \end{cases}. Does \displaystyle\lim_{x \to 2} g(x) exist?
Left-hand limit — approaching from below, so use the x<2 piece: \lim_{x \to 2^-} g(x) = \lim_{x \to 2^-} (x+1) = 2 + 1 = 3
Right-hand limit — approaching from above, so use the x\geq2 piece: \lim_{x \to 2^+} g(x) = \lim_{x \to 2^+} (x^2-1) = 2^2 - 1 = 3
Both sides agree, so \displaystyle\lim_{x \to 2} g(x) = 3 — and since g(2) = 2^2-1 = 3 too, g is also continuous at x=2
Evaluate \displaystyle\lim_{x \to 2} \frac{x^2 - 4}{x - 2}.
Direct substitution gives \frac{0}{0} (indeterminate), so factor first:
\frac{x^2-4}{x-2} = \frac{(x-2)(x+2)}{x-2} \quad \Rightarrow \quad \lim_{x \to 2} (x+2) = 4
Practice Time!
Next session:
Maths Boot Camp