Regression Methods and Regularization

Lecture 1

 

Stacks Image 48736

 

[ view ]

 
CLASSIC CURVE FITTING METHODS: The basic theory of curve fitting and least-square error is developed.

 

Lecture 2

 

Stacks Image 47912

 

[ view ]

 
NONLINEAR REGRESSION AND GRADIENT DESCENT: Derivative-based methods are some of the work-horse algorithms of modern optimization, including gradient descent.

 

Lecture 3

 

Stacks Image 47931

 

[ view ]

 
OVER- AND UNDER-DETERMINED SYSTEMS: We consider regression on classical Ax=b problems in the over- and under-determined scenarios. Regularizers are considered.

 

 

 

Lecture 4

 

Stacks Image 48830

 

[ view ]

 
OPTIMIZATION AS THE CORNERSTONE OF REGRESSION: The role of regularization on regression is considered in the context of an associated optimization formulation.

 

Lecture 5

 

Stacks Image 48843

 

[ view ]

 
PARETO FRONTS: The concept of the Pareto frontier and Pareto optimal solutions is considered along with parsimony of solutions.

 

Lecture 6

 

Stacks Image 48856

 

[ view ]

 
MODEL SELECTION: CROSS-VALIDATION: The critical concept of cross-validation is considered for establishing models that are not over-fit.

 

 

 

Lecture 7

 

Stacks Image 48962

 

[ view ]

 
MODEL SELECTION: INFORMATION CRITERIA: The AIC and BIC in evaluating the relative fit of models is considered.

 

© 2016 kutz