Beginning Scientific Computing

Optimization

Section 4.3: Over- and Under-determined Systems

Stacks Image 4593

 

[ View ]

Section 4.4: Optimization for Regressions

Stacks Image 4584

 

[ View ]

 

Section 6.3: The Backpropagation Algorithm

Stacks Image 4620

 

[ View ]

Section 6.4: The Stochastic Gradient Descent Algorithm

Stacks Image 4611

 

[ View ]

 

 

 

 

KEY REFERENCES AND SUPPLEMENTARY VIDEOS

 

Lecture 1: Ch. 5.1

 

Stacks Image 4676

 

[ view ]

 
UNCONSTRAINED OPTIMIZATION (DERIVATIVE-FREE METHODS): We introduce some of the basic techniques of optimization that do not require derivative information from the function being optimized, including golden section search and successive parabolic interpolation.

 

MATLAB COMMANDS

FMINBND

MATLAB CODE

 

Lecture 2: Ch. 5.2

 

Stacks Image 4657

 

[ view ]

 
UNCONSTRAINED OPTIMIZATION (DERIVATIVE METHODS): Derivative-based methods are some of the work-horse algorithms of modern optimization, including gradient descent.

 

MATLAB COMMANDS

FMINSEARCH

MATLAB CODE

 

Lecture 3: Ch. 5.3-5.5

 

Stacks Image 4638

 

[ view ]

 
LINEAR PROGRAMMING AND GENETIC ALGORITHMS: We consider a number of more advanced optimization algorithms that include the genetic algorithm and linear programming for constrained optimization.

 

MATLAB COMMANDS

LINPROG GA

MATLAB CODE

 

© 2015 kutz