Lecture 4: nonlinear programming

Goal: necessary and sufficient conditions for local optimality in unconstrained nonlinear programs (NLP).

References

Pages 29–41: Stengel, R. F. Optimal Control and Estimation. (Dover, New York, 1994).
Chapter 1: Lewis, F. L., Vrabie, D. & Syrmos, V. L. Optimal Control. (John Wiley & Sons, 2012).

Problem formulation

We want to minimize a given cost function \(c:\mathbb{R}^m\rightarrow\mathbb{R}:u\mapsto c(u)\), i.e. find \(u^*\in\mathbb{R}^m\) such that \(\forall u \neq u^* : c(u^*) < c(u)\). The problem

\[ \min_u\in\mathbb{R}^m c(u) \]

is termed a nonlinear program (NLP).

Gradient descent

Idea: starting from \(u\in\mathbb{R}^m\) where \(Dc(u) \neq 0\), choosing \(u^+ = u - \gamma Dc(u)\) yields \(c(u^+) < c(u)\) for all \(\gamma > 0\) sufficiently small. Where does this iteration go?

As an illustration, consider the cost function

\[ c(u) = \frac{1}{6} u^6 - \frac{7}{5} u^5 + \frac{17}{4} u^4 - \frac{17}{3} u^* + 3 u^2 \]

shown in the following figure.

Code

from imports import *

def c(u):
    return u**6/6 - 7*u**5/5 + 17*u**4/4 - 17*u**3/3 + 3*u**2

def Dc(u):
    return u**5 - 7*u**4 + 17*u**3 - 17*u**2 + 6*u

def D2c(u):
    return 5*u**4 - 28*u**3 + 51*u**2 - 34*u + 6

u = np.linspace(-0.5, 3.5, 500)

fig, axes = plt.subplots(3, 1, figsize=(5, 6), sharex=True)

axes[0].plot(u, c(u))
axes[0].axhline(0, color='k', lw=0.5)
axes[0].set_ylabel('$c(u)$')
axes[0].set_title('Cost function and its derivatives')

axes[1].plot(u, Dc(u), color='tab:orange')
axes[1].axhline(0, color='k', lw=0.5)
axes[1].set_ylabel('$Dc(u)$')

axes[2].plot(u, D2c(u), color='tab:green')
axes[2].axhline(0, color='k', lw=0.5)
axes[2].set_ylabel('$D^2c(u)$')

# Mark stationary points
for u_stat in [0, 1, 2, 3]:
    axes[0].plot(u_stat, c(u_stat), 'ro', ms=5)
    axes[1].plot(u_stat, 0, 'ro', ms=5)
    axes[2].plot(u_stat, D2c(u_stat), 'ro', ms=5)

# Label stationary points on the shared x-axis
axes[2].set_xticks([0, 1, 2, 3])
axes[2].set_xticklabels(['$u_1$', '$u_2$', '$u_3$', '$u_4$'], fontsize=12)

plt.tight_layout()
show_with_alt(fig,
    "Three vertically aligned subplots sharing a horizontal u-axis from -0.5 to 3.5. "
    "Top: cost c(u), a degree-6 polynomial with two local minima (u=0, c=0) and (u=3, c=-0.45) "
    "and a local maximum near u=2. "
    "Middle: first derivative Dc(u), zero at stationary points u=0,1,2,3. "
    "Bottom: second derivative D2c(u), positive at u=0 and u=3 (minima), negative at u=2 (maximum), "
    "zero at u=1 (inflection). Red dots mark all four stationary points in each subplot.")

Three vertically aligned subplots sharing a horizontal u-axis from -0.5 to 3.5. Top: cost c(u), a degree-6 polynomial with two local minima (u=0, c=0) and (u=3, c=-0.45) and a local maximum near u=2. Middle: first derivative Dc(u), zero at stationary points u=0,1,2,3. Bottom: second derivative D2c(u), positive at u=0 and u=3 (minima), negative at u=2 (maximum), zero at u=1 (inflection). Red dots mark all four stationary points in each subplot.

Stationary points

Note that there are four points where \(Dc(u) = 0\): \(u_1\), \(u_2\), \(u_3\), \(u_4\) – we term these stationary points. A stationary point can be a (local) minimizer as with \(u_1\) and \(u_4\), (local) maximizer as with \(u_3\), or a saddle (commonly called an inflection point in 1D) as with \(u_2\).

Definition: \(u^*\in\mathbb{R}^m\) is a local minimum of \(c\) if there is an open neighborhood \(U\subset\mathbb{R}^m\) such that \(\forall u \in U : c(u^*) \leq c(u)\). The minimum is strict if the inequality is strict.

Second-order information

Importantly, the second derivative (Hessian matrix) can help distinguish between these three categories: if \(D^2 c(u) > 0\) the stationary point is a local minimizer and it’s a local maximizer if \(D^2 c(u) < 0\). If \(D^2 c(u) = 0\) in 1D or, in higher dimensions, if \(0\) is an eigenvalue of \(D^2 c(u)\), we cannot distinguish between the possible categories and would need higher-order information to do so.

However, cost functions that have definite Hessians are generic, meaning that it is “unlikely” you will encounter a cost function with indefinite Hessian in practice. So the techniques we develop based on first- and second-order data are widely applicable.

Recall: if \(c\) is twice continuously differentiable then \(D^2 c(u)\in\mathbb{R}^{m\times m}\) is symmetric because for all \(i,j\in\left\{1,\dots,m\right\}:\)

\[ D^2 c = \left[ \frac{\partial\ }{\partial u_i} \frac{\partial\ }{\partial u_j} c \right] = \left[ \frac{\partial\ }{\partial u_j} \frac{\partial\ }{\partial u_i} c \right]. \]

Recall: if \(S = S^\top\) then all eigenvalues of \(S\) are real. In this case, \(S > 0 \iff\) all eigenvalues of \(S\) are positive.

Sufficient conditions

Theorem (sufficient conditions for optimality): a stationary point \(u_0\in\mathbb{R}^m\) is a strict local minimum of \(c\) if \(D^2 c(u_0)\) is positive-definite: \(D^2 c(u_0) > 0\).

Q: determine \(c\) such that \(u_0 = 0\) is a local minimum but \(D^2 c(u_0) \neq 0\) (or doesn’t exist!). > \(c(u) = 0\), \(c(u) = u^4\), or \(c(u) = |u|\) all work.

Necessary conditions

Theorem (necessary conditions for optimality): if \(u_0\in\mathbb{R}^m\) is a local minimum of \(c\) then: * if \(c\) is continuously differentiable (\(c\in C^1\)) then \(Dc(u_0) = 0\); * if \(c\) is twice continuously differentiable (\(c\in C^2\)) then \(D^2c(u_0) \geq 0\).

Q: determine necessary conditions on \(b^\top\in\mathbb{R}^{1\times m}\), \(C^T = C\in\mathbb{R}^{m\times m}\) for \(u_0\in\mathbb{R}^m\) to be a local minimum of

\[ c(u) = c(u_0) + b^\top (u - u_0) + \frac{1}{2} (u - u_0)^\top C (u - u_0). \]

\(Dc(u) = b^\top + (u - u_0)^\top C\) so it’s necessary that \(Dc(u_0) = b^T = 0\);

\(D^2 J(u) = C\) so it’s necessary that \(D^2c(u_0) = C \geq 0\).

Q: use the necessary conditions for optimality to find \(u_0\) that minimizes the quadratic cost

\[ c(u) = b^\top u + \frac{1}{2} u^\top C u \]

where \(C = C^\top > 0\).

Necessarily \(Dc(u_0) = b^\top + u_0^\top C = 0 \iff u_0 = - C b\).