Nonlinear Equations using the Newton-Raphson Method

Consider the single, nonlinear equation to be solved for x.

f(x) = 0

The Newton-Raphson method is based on a Taylor series of the equation about the k-th iterate.

We then neglect the second and higher-order terms, set f(xk+1) = 0, and rearrange to get

This method converges provided certain inequalities are satisfied (theorem). In practice the method may not converge unless the initial guess is good, or it may converge for some parameters and not others. Unfortunately, when the method is non-convergent the results look just like there was a mistake in the computer programming; it is hard to distinguish between these situations so that careful programming and testing is required. If the method converges the difference between successive iterates will go something like 0.1, 0.01, 0.0001, 10-8. The error (when it is known) goes in the same way; the method is said to be quadratically convergent when it converges. The method is not robust, since it can fail for poor initial guesses.

If the derivative is difficult to calculate a numerical approximation may be used.

In the secant method the same formula is used as for the Newton-Raphson method, except that the derivative is approximated using the values from the last two iterates.

This is equivalent to drawing a straight line through the last two iterate values on a plot of f(x) versus x. The Newton-Raphson method is equivalent to drawing a straight line tangent to the curve at the last x. In the method of false position (or regula falsi), the secant method is used to get xk+1, but the previous value is taken as either xk-1 or xk. The choice is made so that the function evaluated for that choice has the opposite sign as f( xk+1). This method is slower than the secant method but is more robust and it keeps the root between two points at all times. In all these methods one must watch for bounds on the function and when df/dx = 0 and have appropriate strategies for those cases.

As an example, consider the very simple problem

f(x) = x2 ­ 2

with the solution x = 1.4142136. The derivative is 2x, and the Newton-Raphson method is

Starting with an initial guess of x = 1.0 gives the results shown in the figure.

Figure NR1. Successive Substitution and Newton-Raphson Methods Applied to Example, f(x) = x2 - 2.

Take Home Message: Convergence of the Newton-Raphson Method is very fast, but it does require calculating the derivative.