How do you perform an LU factorization of the matrix
$$ A = \begin{bmatrix} 0 & 1 & 1 \\ 1 & 1 & 1\\ -1 & 1 & -1 \end{bmatrix}? $$If we perform the row operation $R_1 \leftrightarrow R_2$ first
$$ A_1 = \begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1\\ -1 & 1 & -1 \end{bmatrix} $$Let's not do any more row operations:
$$ A_1 =\begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0\\ -1 & 0 & 1 \end{bmatrix}\begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1\\ 0 & 2 & 0 \end{bmatrix}$$$$ R_3 - 2 R_2 \to R_3$$$$ A_1 =\begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0\\ -1 & 2 & 1 \end{bmatrix}\begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1\\ 0 & 0 & -2 \end{bmatrix}$$But, if we want to factorize $A$, how is $A_1$ related to $A$? It follows that all row operations are linear operations:
If you do a row operation on a vector $v = \alpha v_1 + \beta v_2$ to get a vector $\hat v$ it is the same as doing the row operation on $v_1$ and $v_2$ to get $\hat v_1$ and $\hat v_2$, respectively, and then $\hat v = \alpha \hat v_1 + \beta \hat v_2$.
This is important because it implies that all row operations have a matrix representation. We've already found the matrix $L$ for $R_m + \alpha R_p \to R_m$ for $m > p$:
$$L = (l_{ij}), \quad l_{ij} = \begin{cases} 1, & i =j\\ \alpha,& j = m, ~~ i = p,\\ 0, & \text{otherwise}.\end{cases}, ~~\quad L^{-1} = (l'_{ij}), \quad l'_{ij} = \begin{cases} 1, & i =j\\ -\alpha,& j = m, ~~ i = p,\\ 0, & \text{otherwise}.\end{cases}$$To find the matrix representation for a (linear) transformation, you apply the transformation to each of the standard (basis) unit vectors:
$$\begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \to \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} $$$$\begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \to \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} $$$$\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \to \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} $$We assemble the resulting columns into a matrix. The $R_1 \leftrightarrow R_2$ matrix is
$$ P = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix}$$This is called a simple permutation matrix because it corresponds to ONE row interchange. Another way of constructing the matrix associated with a row operation is:
To construct the matrix associated with a row operation, apply that row operation to the identity matrix.
A general simple permutation matrix corresponding to $R_m \leftrightarrow R_k$ is given by
$$ P = (p_{ij}), \quad p_{ij} = \begin{cases}1, & i = j \text{ and } i \neq m,p \text{ and } j \neq m,p\\ 1, & i = m, j = p \text{ or } i = p, j = m\\ 0, & \text{otherwise}.\end{cases}$$It follows immediately, that $p_{ij} = p_{ji}$ so that $P^T = P$.
Returning to our example,
$$ A = \begin{bmatrix} 0 & 1 & 1 \\ 1 & 1 & 1\\ -1 & 1 & -1 \end{bmatrix}. $$$$ PA = A_1 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0\\ -1 & 2 & 1 \end{bmatrix}\begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1\\ 0 & 0 & -2 \end{bmatrix} = LU$$$$ A = P^{-1} LU$$It follows that $P^{-1} = P= P^T$.
Consider a general permutation matrix, which we define to be the product of a number of simple permulation
$$P = P_k P_{k-1} \cdots P_{2} P_{1}.$$Notice that $P^T = P_1^T P_2^T \cdots P_{k-1}^T P_k^T = P_1^{-1} P_2^{-1} \cdots P_{k-1}^{-1} P_k^{-1} = P^{-1}$.
Given an invertible matrix $A$, we want to find a permutation matrix $P$ such that
$$PA = LU$$$$A = P^T LU$$where $L$ is a lower-triangular matrix with ones on the diagonal and $U$ is upper trianguler. Note that such a factorization is NOT unique. Different pivoting strategies give different permutation matrices.
Compute the permuted $LU$ factorization for $A$ using partial pivoting
$$ A = \begin{bmatrix} 0 & 1 & 1 \\ 1 & 1 & 1\\ -1 & 1 & -1 \end{bmatrix} $$You should check the following. If $P_1A$ swaps rows $i$ and $j$ of the matrix $A$ then $AP_1$ swaps columns $i$ and $j$ of $A$.
To compute this factorization, we first write
$$ A = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} 0 & 1 & 1 \\ 1 & 1 & 1\\ -1 & 1 & -1 \end{bmatrix} $$These two identity matrices will be used to capture both the $P$ and $L$ in the matrix factorization
We first must swap the rows one and two (following partial pivoting). Let $P_1$ be the matrix that corresponds to this. Because $P_1 = P_1^{-1}$ we can write:
$$ A = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} P_1 P_1 \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} P_1 P_1 \begin{bmatrix} 0 & 1 & 1 \\ 1 & 1 & 1\\ -1 & 1 & -1 \end{bmatrix} $$The matrix $P_1 I P_1 = I$. As we progress, this will not always happen. Recall that multiplication on the right by $P_1$ corresponds to column swaps:
$$ A = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1\\ -1 & 1 & -1 \end{bmatrix} $$Now we perform the row operation $R_3 + R_1 \to R_3$. This modifies the matrix that will become $L$ in the $A = P^T LU$ factorization
$$ A = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -1 & 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1\\ 0 & 2 & 0 \end{bmatrix} $$Following the rules of partial pivoting, we must swap rows 2 and 3. Let $P_2$ be the matrix that does this:
$$ A = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix} P_2 P_2 \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -1 & 0 & 1 \end{bmatrix} P_2P_2 \begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1\\ 0 & 2 & 0 \end{bmatrix} $$We have one final row operation: $R_3 - 1/2 R_2 \to R_3$
$$ A = \begin{bmatrix} 0 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ -1 & 1 & 0 \\ 0 & 1/2 & 1 \end{bmatrix} \begin{bmatrix} 1 & 1 & 1 \\ 0 & 2 & 0\\ 0 & 0 & 1 \end{bmatrix} $$Naive Gaussian elimination: $$A = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0\\ -1 & 2 & 1 \end{bmatrix}\begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1\\ 0 & 0 & -2 \end{bmatrix} = P^T LU$$
Partial pivoting: $$ A = \begin{bmatrix} 0 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ -1 & 1 & 0 \\ 0 & 1/2 & 1 \end{bmatrix} \begin{bmatrix} 1 & 1 & 1 \\ 0 & 2 & 0\\ 0 & 0 & 1 \end{bmatrix} = P^TLU$$
The entries of $L$ for partial pivoting will always be less than one!