mmtom

Numerical MethodsA Manual Lektor: Youri V. Shestopalov e-mail: [email protected] Tel. 054-7001856 Homepage: www.ingvet.kau.se\ ∼youri Karlstads Universitet 2000 1 Contents 1 Numerical Methods in General 3 1.1 Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Formulas for errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Error propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Loss of significant digits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 Binary number system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2 Solution of Nonlinear Equations 6 2.1 Fixed-point iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Convergence of fixed-point iterations . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Newton’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.4 Secant method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.5 Numerical solution of nonlinear equations using MATLAB . . . . . . . . . . . . 11 3 Interpolation 13 3.1 Quadratic interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.2 Estimation by the error principle . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3 Divided differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4 Numerical Integration and Differentiation 29 4.1 Rectangular rule. Trapezoidal rule . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.2 Error bounds and estimate for the trapezoidal rule . . . . . . . . . . . . . . . . 30 4.3 Numerical integration using MATLAB . . . . . . . . . . . . . . . . . . . . . . . 32 4.4 Numerical differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5 Approximation of Functions 33 5.1 Best approximation by polynomials . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.2 Chebyshev polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 6 Numerical Methods in Linear Algebra. Gauss Elimination 42 6.1 Gauss elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 6.2 Application of MATLAB for solving linear equation systems . . . . . . . . . . . 46 7 Numerical Methods in Linear Algebra. Solution by Iteration 49 7.1 Convergence. Matrix norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 7.2 Jacobi iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 7.3 Gauss–Seidel iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 7.4 Least squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 8 Numerical Methods for First-Order Differential Equations 54 8.1 Euler method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 8.2 Improved Euler method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 8.3 Runge–Kutta methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 8.4 Numerical solution of ordinary differential equations using MATLAB . . . . . . 59 2 1 Numerical Methods in General Significant digit (S) of a number c is any given digit of c except possibly of zeros to the left of the firt nonzero digit: each of the numbers 1360. 1.360, 0.001360 has 4 significant digits. In a fixed-point system all numbers are given with a fixed number of decimal places: 62.358, 0.013, 1.000. In a floating-point system the number of significant digits is fixed and the decimal point is floating: 0.6238 ×10 3 = 6.238 ×10 2 = 0.06238 ×10 4 = 623.8, 0.1714 ×10 −13 = 17.14 ×10 −15 = 0.01714 ×10 −12 = 0.00000000000001714 (13 zeros after the decimal point) −0.2000 ×10 1 = −0.02000 ×10 2 = −2.000, also written 0.6238E03 0.1714E-13, −0.2000E01. Any number a can be represented as a = ±m· 10 e , 0.1 ≤ m < 1, e integer. On the computer m is limited to t digits (e.g., t = 8) and e is also limited: a = ± m· 10 e , m = 0.d 1 d 2 . . . d t , d 1 > 0, |e| < M. The fractional part m (or m) is called the mantissa and e is called the exponent. The IEEE Standard for single precision: −38 < e < 38. 1.1 Rounding 3.45 ≈ 3.4, 3.55 ≈ 3.6 to 1 decimal [2S (significant digits)] 1.2535 ≈ 1.254 to 3 decimals (4S), 1.2535 ≈ 1.25 to 2 decimals (3S), 1.2535 ≈ 1.3 to 1 decimal (2S), but 1.25 ≈ 1.2 to 1 decimal (2S), 1.2 Formulas for errors If ˜ a is an approximate value of a quantity whose exact value is a then = a − ˜ a is called the error of a. Hence a = ˜ a + For example, if ˜ a = 10.5 is an approximation of a = 10.2, its error is = −0.3. 3 The relative error r of ˜ a r = a = a − ˜ a a = Error True value (a = 0). r ≈ ˜ a . The error bound for ˜ a is a number β such that || ≤ β hence |a − ˜ a| ≤ β. Similarly, for the relative error, the error bound is a number β r such that | r | ≤ β r hence ¸ ¸ ¸ ¸ a − ˜ a a ¸ ¸ ¸ ¸ ≤ β r . 1.3 Error propagation In addition and subtraction, an error bound for the results () is given by the sum of the error bounds for the terms (β 1 and β 2 ): || ≤ β 1 + β 2 . In multiplicaion and division, a bound for the relative error of the results ( r ) is given (ap- proximately) by the sum of the bounds for the relative errors of the given numbers (β r1 and β r2 ): | r | ≤ β r1 + β r2 . 1.4 Loss of significant digits Consider first the evaluation of f(x) = x[ √ x + 1 − √ x] (1) for an increasing x using a six-digit calculator (6S computations): ———————————————————————– x computed f(x) true f(x) ———————————————————————– 1.00000 .414210 .414214 10.0000 1.54430 1.54347 100.000 4.99000 4.98756 1000.00 15.8000 15.8074 10000.0 50.0000 49.9988 100000 100.000 158.113 ———————————————————————– In fact, in 6S computations, we have, for x = 100, √ 100 = 10.0000, √ 101 = 10.0499. (2) 4 The first value is exact, and the second value is correctly rounded to six significant digits (6S). Next √ x + 1 − √ x = √ 101 − √ 100 = 0.04990 (3) The true value should be .049876 (rounded to 6D). The calculation performed in (3) has a a loss- of-significance error: three digits of accuracy in √ x + 1 = √ 101 were canceled by subtraction of the corresponding digits in √ x = √ 100. The properly reformulated expression f(x) = x √ x + 1 − √ x 1 · √ x + 1 + √ x √ x + 1 + √ x = x √ x + 1 + √ x (4) will not have loss-of-significance error, because we have no subtraction. Completing the 6S computation by (4) we obtain the result f(100) = 4.98756 which is true to six digits. 1.5 Binary number system In the decimal system (with the base 10 and allowed digits 0, 1, ..., 9), a number, for example, 342.105, means 3 · 10 2 + 4 · 10 1 + 2 · 10 0 + 1 · 10 −1 + 0 · 10 −2 + 5 · 10 −3 (5) In the binary system (with the base 2 and allowed digits 0, 1), a number, for example, 1101.11 means 1 · 2 3 + 1 · 2 2 + 0 · 2 1 + 1 · 2 0 + 1 · 2 −1 + 1 · 2 −2 (6) in the decimal system. We write also (1101.11) 2 = (13.75) 10 because 1 · 2 3 + 1 · 2 2 + 0 · 2 1 + 1 · 2 0 + 1 · 2 −1 + 1 · 2 −2 = 8 + 4 + 1 + 0.5 + 0.25 = 13.75. To perform the conversion from the decimal to binary system of a number a n · 2 n + a n−1 · 2 n−1 + . . . + a 1 · 2 1 + a n · 2 0 = x (7) or (a n a n−1 . . . a 1 a 0 ) 2 = (x) 10 divide x by 2 and denote the quotient by x 1 ; the reminder is a 0 . Next divide x 1 by 2 and denote the quotient by x 2 ; the reminder is a 1 . Continue this process to find a 2 , a 3 , . . . a n in succession. 5 Problem 1.1 Floating-point form of the numbers 23.49, −302.867, 0.000527532, −0.25700 rounded to 4S (significant digits): 0.2349 ×10 2 , −0.3029 ×10 3 , 0.5275 ×10 −3 , −0.2570 ×10 5 also written 0.2349E02, −0.3029E03, 0.5275E-3, −0.2570E00. Problem 1.2 Compute a b −c = a(b + c) b 2 −c 2 , with a = 0.81534, b = 35.724, c = 35.596. and different types of rounding. We calculate R = a(b + c) b 2 −c 2 step by step with 5S: b + c = 71.320, a(b + c) = 58.150, b 2 = 1276.2, c 2 = 1267.1, b 2 −c 2 = 9.1290, R = 58.150 9.1290 = 6.3698 Round to 4S, 3S, and 2S to obtain R = 58.15 9.129 = 6.370, R = 58.2 9.13 = 6.37, R = 58 9.1 = 6.4, R = 60 10 = 6. 2 Solution of Nonlinear Equations There are some equations of nonlinear type for which there exist analytical methods leading to a formula for the solution. The quadratic equations or trigonometric equations such as x 2 − 3x + 2 = 0, sin x + cos x = 1 provide simple examples. Among them is of course a linear equation ax + b = 0. However, many nonlinear equations, which may be represented in the general form f(x) = 0 (8) where f(x) is assumed to be an arbitrary function of one variable x, cannot be solved directly by analytical methods and a numerical method based on approximation must be used. Each numerical method for the solution of nonlinear equation (8) has the form of an algo- rithm which produces a sequence of numbers x 0 , x 1 , . . . approaching a root of (8). In this case we say that the method is convergent. A method must contain a rule for the stop, when we are close enough to the true value of the solution. 6 But usually the true solution is not known and this rule may be chosen in the form of simultaneous fulfilment of two conditions | f(x N ) | < f ; (9) |x N − x N−1 | < x (10) at a certain step with number N = 1, 2, . . ., where f and x are given and they indicate the required accuracy of the method. The value x 0 plays a special role and it is called initial approximation. 2.1 Fixed-point iteration Consider the equation f(x) = 0 and assume that we can rewrite it in the equivalent form x = g(x). Then the fixed-point iterations for finding roots of x = g(x) is defined by x n+1 = g(x n ) (11) Consider solving the equation f(x) = x 2 −5 to find the root a = √ 5 = 2.2361 (5S). List four types of fixed-point iterations: i x n+1 = g 1 (x n ) = 5 + x n −x 2 n ii x n+1 = g 2 (x n ) = 5 x n iii x n+1 = g 3 (x n ) = 1 + x n − 1 5 x 2 n iv x n+1 = g 4 (x n ) = 1 2 _ x n + 5 x n _ ————————————————————- n x n ,i x n ,ii x n ,iii x n ,iv ————————————————————- 0 2.5 2.5 2.5 2.5 1 1.25 2.0 2.25 2.25 2 4.6875 2.5 2.2375 2.2361 3 −12.2852 2.0 2.2362 2.2361 ———————————————————— 2.2 Convergence of fixed-point iterations Example 2.1 Consider the equation f(x) = x 2 −3x + 1 7 written in the form x = g 2 (x) = 3 − 1 x . Solve the equaiton x = g 2 (x) by the fixed-point iteration process. Setting x 0 = 1 we obtain x 1 = g 2 (x 0 ) = 3 − 1 1 = 2, x 2 = g 2 (x 1 ) = 3 − 1 2 = 2.5, x 3 = g 2 (x 2 ) = 3 − 1 2.5 = 3 − 2 5 = 13 5 = 2.6, x 4 = g 2 (x 3 ) = 3 − 1 2.6 = 2.615, . . . so that 1 = x 0 < x 2 < x 3 < . . . hence 1 = 1 x 0 > 1 x 1 > 1 x 2 . . . and 1 x n ≤ K = 1 2 , n = 1, 2, . . . . Let us perform a stepwise estimation of the difference |x n+1 −x n | and then of the error |x n+1 −s| where s = g(s) is the desired solution, taking into account that x n+1 = g(x n ), n = 0, 1, 2, . . .: |x 2 −x 1 | = |g 2 (x 1 ) −g 2 (x 0 )| = ¸ ¸ ¸ ¸ 3 − 1 x 1 −3 + 1 x 0 ¸ ¸ ¸ ¸ = ¸ ¸ ¸ ¸ 1 x 0 − 1 x 1 ¸ ¸ ¸ ¸ = |x 0 −x 1 | |x 0 ||x 1 | = 1 5 |x 1 −x 0 |; |x 3 −x 2 | = |g 2 (x 2 ) −g 2 (x 1 )| = ¸ ¸ ¸ ¸ 1 x 1 − 1 x 2 ¸ ¸ ¸ ¸ = |x 2 −x 1 | |x 2 ||x 1 | = 1 2.6 · 2.5 |x 2 −x 1 | = 1 6.5 |x 2 −x 1 | = 1 6.5 1 5 |x 1 −x 0 | = 1 30.25 |x 1 −x 0 |; . . . ; |x n+1 −x n | = p 1 · p 2 · . . . · p n |x 1 −x 0 |, 0 < p i ≤ K < 1, n = 0, 1, 2, . . . , in summary, |x n+1 −x n | ≤ K n |x 1 −x 0 | = 1 2 n |x 1 −x 0 | _ = 1 2 n , x 0 = 1 _ , n = 0, 1, 2, . . . . On the other hand, |x n −s| = |g 2 (x n−1 ) −g 2 (s)| = ¸ ¸ ¸ ¸ ¸ 1 s − 1 x n−1 ¸ ¸ ¸ ¸ ¸ = |s −x n−1 | |s||x n−1 | ≤ K|x n−1 −s| with K ≤ 1 2 because from the crude analysis of the considered function f(x) = x 2 −3x +1 we know that s > 2 (indeed, f(2) = −1 < 0 and f(3) = 2 > 0) and x n > 2. Proceeding in a similar manner, we obtain |x n −s| ≤ K|x n−1 −s| = K|g 2 (x n−2 ) −g 2 (s)| =≤ K 2 |x n−2 −s| ≤ . . . ≤ K n |x 0 −s|. Since K < 1, we have K n →0 and |x n −s| →0 as n →∞. The same proof can be obtained if we apply on each step above the mean value theorem of calculus, according to which g 2 (x) −g 2 (s) = g 2 (t)(x −s) = 1 t 2 (x −s), t between x and s 8 and take into account that we seek for roots in the domain t ≥ k > 1 where g 2 (t) = 1 t 2 ≤ ˜ K < 1. The same proof is valid also in a rather general case. Let us formulate the general statement concerning the convergence of fixed-point iterations. Let x = s be a solution of x = g(x) and suppose that g(x) has a continuous derivative in some interval J containing s. Then if |g (x)| ≤ K < 1 in J, the iteration process defined by x n+1 = g(x n ) converges for any x 0 in J. In the above-considered case g(x) = g 2 (x) = 3 − 1 x and one may take as J an interval x : k 1 < x < k 2 for any k 1 and k 2 such that k 1 > 1 and k 2 > k 1 because in every such interval g (x) = 1 x 2 ≤ K = 1 k 2 1 < 1. 2.3 Newton’s method Consider the graph of a function y = f(x). The root a occurs where the graph crosses the x-axis. We will usually have an estimate of a (which means that we have some preliminary information about the position of a root), and it will be denoted by x 0 . To improve on this estimate, consider the straight line that is tangent to the graph at the point (x 0 , f(x 0 )). If x 0 is near a, this tangent line should be nearly coincident with the graph of y = f(x) for points x about a. Then the root of the tangent line should nearly equal a. This root is denoted here by x 1 . To find a formula for x 1 , consider the slope of the tangent line. Using the derivative f (x) we know from calculus that the slope of the tangent line at (x 0 , f(x 0 )) is f (x 0 ). We can also calculate the slope using the fact that the tangent line contains the two points (x 0 , f(x 0 )) and (x 1 , 0). This leads to the slope being equal to f(x 0 ) − 0 x 0 − x 1 , namely, the difference in the y-coordinates divided by the difference in the x-coordinates. Equat- ing the two different formulas for the slope, we obtain f (x 0 ) = f(x 0 ) x 0 − x 1 . This can be solved to give x 1 = x 0 − f(x 0 ) f (x 0 ) . 9 Since x 1 should be an improvement over x 0 as an estimate of a, this entire procedure can be repeated with x 1 as the initial approximation. This leads to the new estimate, or, in other words, to the new step of the algorithm x 2 = x 1 − f(x 1 ) f (x 1 ) . Repeating this process, we obtain a sequence of numbers x 1 , x 2 , x 3 , . . . that we hope approach the root a. These numbers may be called iterates, and they are defined by the following general iteration formula x n+1 = x n − f(x n ) f (x n ) , n = 0, 1, 2, . . . (12) which defines the algorithm of Newton’s method for solving the equation f(x) = 0. Note that actually Newton’s method is a particular case of fixed-point iterations x n+1 = g(x n ), n = 0, 1, 2, . . . with g(x n ) = x n − f(x n ) f (x n ) . (13) Example 2.2 Let us solve the equation f(x) = x 6 − x + 1, f (x) = 6x 5 − 1. The iteration of Newton’s method is given by x n+1 = x n − x 6 n − x n − 1 6x 5 n − 1 , n = 0, 1, 2, . . . . (14) We use an initial approximation of x 0 = 1.5. The results are shown in the table below. The column “x n − x n−1 ” is an estimate of the error a − x n−1 ———————————————————————– x n f(x n ) x n − x n−1 ———————————————————————– 1.5 88.9 1.30049088 25.40 −0.200 1.18148042 0.538 −0.119 1.13945559 0.0492 −0.042 1.13477763 0.00055 −.00468 1.13472415 0.000000068 −0.0000535 1.13472414 −0.0000000040 −0.000000010 ———————————————————————– The true root is a = 1.134724138 and x 6 equals a to nine significant digits. Note that the change of sign in the last line of the table indicates the presence of root in the interval (x 5 , x 6 . We see that Newton’s method may converge slowly at first; but as the iterates come closer to the root, the speed of convergence increases, as shown in the table. If to take, for example, f = 10 −8 = 0.00000001 and x = 10 −6 = 0.000001, the method stops, according to the rule (2), at the step N = 6. 10 2.4 Secant method Take Newton’s method (12) and to approximate the derivative by the formula f (x) ≈ f(x n ) −f(x n−1 ) x n −x n−1 . We get the formula for the secant method x n+1 = x n − f(x n ) x n −x n−1 f(x n ) −f(x n−1 ) (15) = x n−1 f(x n ) −x n f(x n−1 ) f(x n ) −f(x n−1 ) In the secant method the points are used in strict sequence. As each new point is found the lowest numbered point in the sequence is discarded. In this method it is quite possible for the sequence to diverge. 2.5 Numerical solution of nonlinear equations using MATLAB It is very easy to write a MATLAB program for computing a root of the given nonlinear equation f(x) = 0 [function f(x) should be saved in an M-file, e.g., ff.m], written it in the form x = g(x) (16) using the iterations x n+1 = g(x n ), n = 0, 1, 2, . . . . Indeed, it is sufficient to save the function g(x) in (16) in an M-file (e.g., gg.m), choose the initial approximation x0 (according to the preliminary information about positions of roots), set x = x0, and repeat (16) several times. In order to stop, one may use the rule (9) as follows: calculate each time the difference between the current and preceding values of x and stop when this value becomes less than the given tolerance [a small number x in (9); one may choose, e.g., x = 10 −4 ] which specifies the accuracy of calculating the root. In the case of Newton’s method one should use formula (13) to create g(x), that is, to calculate the derivative of the initial function f(x). To find a zero of a function MATLAB uses an appropriate method excecuted by the following MATLAB command: fzero(fcn,x0) returns one of the zeros of the function defined by the string fcn. The initial approximation is x0. The relative error of approximation is the MATLAB predefined variable eps. fzero(fcn,x0,tol) does the same with the relative error tol. 11 Example 2.3 Find the points(s) of intersection of the graphs of functions sinx and 2x −2, i.e., the point(s) at which sin x = 2x −2. To this end, create an M-file sinm.m for the function sinm(x) = sin x −2x + 2 function s = sinm(x) s = sin(x) - 2.∗x + 2; Estimate the position of the initial approximation by plotting functions: fplot(’sinm’, [-10,10]) grid on; title(’The sin(x) - 2.∗x + 2 function’); We see that x0 = 2 is a good initial approximation. Therefore, we may type xzero = fzero(’sinm’,2) to obtain xzero = 1.4987 Problem 2.1 Solve the equation f(x) = x 3 −5.00x 2 + 1.01x −1.88 by the fixed-point iteration. According to the method of fixed-point iteration defined by x n+1 = g(x n ), the equation f(x) = x 3 −5.00x 2 + 1.01x −1.88 is transformed to the approriate form x = g(x) = 5.00x 2 −1.01x −1.88 x 2 The iterations computed by x n+1 = g(x n ), n = 0, 1, 2, . . . , are x 0 = 1, x 1 = 2.110, x 2 = 4.099, x 3 = 4.612, x 4 = 4.6952, x 5 = 4.700; x 0 = 5, x 1 = 4.723, x 2 = 4.702, x 3 = 4.700. Problem 2.2 Find Newton’s iterations for the cube root and calculate 7 1/3 . The desired value 7 1/3 is the root of the equation x 3 −7 = 0. The Newton iterations defined as x n+1 = G(x n ), G(x n ) = x n − f(x n ) f (x n ) , n = 0, 1, 2, . . . 12 give, for f(x) = x 3 −7, f (x) = 3x 2 ; x n+1 = 2x 3 n + 7 3x 2 n , n = 0, 1, 2, . . . . The computed result is x 4 = 1.912931. Check the result using the fzero MATLAB command. Problem 2.3 Use Newton’s method to find the root of the equation x 2 − 2 = 0 with the accuracy x = 0.001 and x = 0.00001. Compare the number of iterations. Draw the graphs. Check the result using the fzero MATLAB command. 3 Interpolation Introduce the basic terms: – interpolation polynomial; – Lagrange interpolation polynomial; – Newton interpolation polynomial; – interpolation points (nodes); – linear interpolation; – piecewise linear interpolation; – deviation; – interpolation error. We shall be primarly concerned with the interpolation of a function of one variable: given a function f(x) one chooses a function F(x) from among a certain class of functions (frequently, but not always, the class of polynomials) such that F(x) agrees (coincides) with f(x) at certain values of x. These values of x are often referred to as interpolation points, or nodes, x = x k ; k = 0, 1, . . .. The actual interpolation always proceeds as follows: the function f(x) to be interpolated is replaced by a function F(x) which a) deviates as little as possible from f(x); b) can be easily evaluated. Assume that it is given a function f(x) defined in an interval a ≤ x ≤ b and its values f(x i ) = f i (ordinates) at n + 1 different nodes x 0 , x 1 , . . . , x n lying on [a, b] are known. We seek to determine a polynomial P n (x) of the degree n which coincides with the given values of f(x) at the interpolation points: P n (x i ) = f i (i = 0, 1, . . . n). (17) The possibility to solve this problem is based on the following theorem: there is exactly one polynomial P n (x) of degree less or equal n which satisfies the conditions (1). 13 For n = 1 when P 1 (x) = Ax + B is a linear function, this statement is proved below. This simplest case corresponds to the linear interpolation by the straight line through two points (x 0 , f 0 ) and (x 1 , f 1 ). Given a function f(x) defined in an interval a ≤ x ≤ b we seek to determine a linear function F(x) such that f(a) = F(a), f(b) = F(b). (18) Since F(x) has the form F(x) = Ax + B for some constants A and B we have F(a) = Aa + B, F(b) = Ab + B. Solving for A and B we get A = f(b) −f(a) b −a , B = bf(a) −af(b) b −a and, by (18), F(x) = f(a) + x −a b −a (f(b) − f(a)). (19) One can verify directly that for each x the point (x, F(x)) lies on the line joining (a, f(a)) and (b, f(b)). We can rewrite (19) in the form F(x) = w 0 (x)f(a) + w 1 (x)f(b), where the weights w 0 (x) = b −x b −a , w 1 (x) = x −a b −a . If x lies in the interval a ≤ x ≤ b, then the weights are nonnegative and w 0 (x) + w 1 (x) = 1, hence, in this interval 0 ≤ w i (x) ≤ 1, i = 0, 1. Obviously, if f(x) is a linear function, then interpolation process is exact and the function F(x) from (19) coincides with f(x): the bold curve and the dot line on Fig. 1 between the points (a, f(a)), (b, f(b)) coincide. The interpolation error (x) = f(x) − F(x) 14 shows the deviation between interpolating and interpolated functions at the given point x ∈ [a, b]. Of course, (a) = (b) = 0 and (x) ≡ 0, x ∈ [a, b] if f(x) is a linear function. In the previous example we had only two interpolation points, x = x 0 = a, x = x 1 = b. Now let us consider the case of linear (piecewise linear) interpolation with the use of several interpolation points spaced at equal intervals: x k = a + kh; k = 0, 1, 2, . . . , M, h = b −a M ; M = 2, 3, . . . . For a given integer M we construct the interpolating function F(M; x) which is piecewise linear and which agrees with f(x) at the M + 1 interpolation points. In each subinterval [x k , x k+1 ] we determine F(M; x) by linear interpolation using formula (19). Thus we have for x ∈ [x k , x k+1 ] F(M; x) = f(x k ) + x −x k x k+1 −x k (f(x k+1 ) −f(x k )). If we denote f k = f(x k ) and use the first forward difference ∆f k = f k+1 −f k we obtain F(M; x) = f k + (x −x k ) ∆f k h , x ∈ [x k , x k+1 ]. One can calculate interpolation error in each subinterval and, obviously, (x k ) = 0, k = 0, 1, 2, . . . M. Example 3.1 Linear Lagrange interpolation Compute ln 9.2 from ln 9.0 = 2.1972 and ln 9.5 = 2.2513 by the linear Lagrange interpolation and determine the error from a = ln 9.2 = 2.2192 (4D). Solution. Given (x 0 , f 0 ) and (x 1 , f 1 ) we set L 0 (x) = x −x 1 x 0 −x 1 , L 1 (x) = x −x 0 x 1 −x 0 , which gives the Lagrange polynomial p 1 (x) = L 0 (x)f 0 + L 1 (x)f 1 = x −x 1 x 0 −x 1 f 0 + x −x 0 x 1 −x 0 f 1 . In the case under consideration, x 0 = 9.0, x 1 = 9.5, f 0 = 2.1972, and f 1 = 2.2513. Calculate L 0 (9.2) = 9.2 −9.5 9.0 −9.5 = 0.6, L 1 (9.2) = 9.2 −9.0 9.5 −9.0 = 0.4, and get the answer ln 9.2 ≈ ˜ a = p 1 (9.2) = L 0 (9.2)f 0 + L 1 (9.2)f 1 = 0.6 · 2.1972 + 0.4 · 2.2513 = 2.2188. The error is = a − ˜ a = 2.2192 −2.2188 = 0.0004. 15 3.1 Quadratic interpolation This interpolation corresponds to that by polynomials of degree n = 2 when we have three (different) nodes x 0 , x 1 , x 2 . The Lagrange polynomials of degree 2 have the form l 0 (x) = (x −x 1 )(x −x 2 ) (x 0 −x 1 )(x 0 −x 2 ) ; l 1 (x) = (x −x 0 )(x −x 2 ) (x 1 −x 0 )(x 1 −x 2 ) ; (20) l 2 (x) = (x −x 0 )(x −x 1 ) (x 2 −x 0 )(x 2 −x 1 ) . Now let us form the sum P 2 (x) = f 0 l 0 (x) + f 1 l 1 (x) + f 2 l 2 (x). (21) The following statements hold: 1) P 2 (x) again is a polynomial of degree 2; 2) P 2 (x j ) = f j , because among all terms, only l j (x j ) = 0; 3) if there is another polynomial Q 2 (x) of degree 2 such that Q 2 (x j ) = f j , then R 2 (x) = P 2 (x)−Q 2 (x) is also a polynomial of degree 2, which vanishes at the 3 different points x 0 , x 1 , x 2 , or, in other words, the quadratic equation R 2 (x) = 0 has three different roots; therefore, necessarily, R 2 (x) ≡ 0. Hence (21), or, what is the same, the Lagrange interpolation formula yields the uniquely determined interpolation polynomial of degree 2 corresponding to the given interpolation points and ordinates. Example 3.2 For x 0 = 1, x 1 = 2, x 2 = 4 the Lagrange polynomials (20) are: l 0 (x) = x −2 1 −2 x −4 1 −4 = 1 3 (x 2 −6x + 8); l 1 (x) = x −1 2 −1 x −4 2 −4 = − 1 2 (x 2 −5x + 4); l 2 (x) = x −1 4 −1 x −2 4 −2 = 1 6 (x 2 −3x + 2). Therefore, P 2 (x) = f 0 3 (x 2 −6x + 8) − f 1 2 (x 2 −5x + 4) + f 2 6 (x 2 −3x + 2). If to take, for example, f 0 = f 2 = 1, f 0 = 0, then this polynomial has the form P 2 (x) = 1 2 (x 2 −5x + 6) = 1 2 (x −2)(x −3) and it coincides with the given ordinates at interpolation points P 2 (1) = 1; P 2 (2) = 0; P 2 (4) = 1. 16 If f(x) is a quadratic function, f(x) = ax 2 + bx + c, a = 0, then interpolation process is exact and the function P 2 (x) from (21) will coincide with f(x). The interpolation error (x) = f(x) − P 2 (x) shows the deviation between interpolation polynomial and interpolated function at the given point x ∈ [x 0 , x 2 ]. Of course, (x 0 ) = (x 1 ) = (x 2 ) = 0 and r(x) ≡ 0, x ∈ [x 0 , x 2 ] if f(x) is a quadratic function. Example 3.3 Quadratic Lagrange interpolation Compute ln 9.2 from ln 9.0 = 2.1972, ln 9.5 = 2.2513, and ln 11.0 = 2.3979 by the quadratic Lagrange interpolation and determine the error from a = ln 9.2 = 2.2192 (4D). Solution. Given (x 0 , f 0 ), (x 1 , f 1 ), and (x 2 , f 2 ) we set L 0 (x) = l 0 (x) l 0 (x 0 ) = (x −x 1 )(x −x 2 ) (x 0 −x 1 )(x 0 −x 2 ) , L 1 (x) = l 1 (x) l 1 (x 1 ) = (x −x 0 )(x −x 2 ) (x 1 −x 0 )(x 1 −x 2 ) , L 2 (x) = l 2 (x) l 2 (x 2 ) = (x −x 0 )(x −x 1 ) (x 2 −x 0 )(x 2 −x 1 ) , which gives the quadratic Lagrange polynomial p 2 (x) = L 0 (x)f 0 + L 1 (x)f 1 + L 2 (x)f 2 . In the case under consideration, x 0 = 9.0, x 1 = 9.5, x 2 = 11.0 and f 0 = 2.1972, f 1 = 2.2513, f 2 = 2.3979. Calculate L 0 (x) = (x −9.5)(x −11.0) (9.0 −9.5)(9.0 −11.0) = x 2 −20.5x + 104.5, L 0 (9.2) = 0.5400; L 1 (x) = (x −9.0)(x −11.0) (9.5 −9.0)(9.5 −11.0) = 1 0.75 (x 2 −20x + 99), L 1 (9.2) = 0.4800; L 2 (x) = (x −9.0)(x −9.5) (11.0 −9.0)(11.0 −9.5) = 1 3 (x 2 −18.5x + 85.5), L 2 (9.2) = −0.0200 and get the answer ln 9.2 ≈ p 2 (9.2) = L 0 (9.2)f 0 + L 1 (9.2)f 1 + L 2 (9.2)f 2 = 0.5400 · 2.1972 + 0.4800 · 2.2513 −0.0200 · 2.3979 = 2.2192, which is exact to 4D. 17 The Lagrange polynomials of degree n = 2, 3 . . . are l 0 (x) = w 0 1 (x)w 0 2 (x) . . . w 0 n (x); l k (x) = w k 0 (x)w k 1 (x) . . . w k k−1 (x)w k k+1 . . . w k n (x), k = 1, 2 . . . , n −1; l n (x) = w n 0 (x)w n 1 (x) . . . w n n−1 (x), where w k j (x) = x −x j x k −x j ; k = 0, 1, . . . n, j = 0, 1, . . . n, k = j. Furthermore, l k (x k ) = 1, l k (x j ) = 0, j = k. The general Lagrange interpolation polynomial is P n (x) = f 0 l 0 (x) + f 1 l 1 (x) + . . . + f n−1 l n−1 (x) + f n l n (x), n = 1, 2, . . . , (22) and it uniquely determines the interpolation polynomial of degree n corresponding to the given interpolation points and ordinates. Error estimate is given by n (x) = f(x) −p n (x) = (x −x 0 )(x −x 1 ) . . . (x −x n ) f n+1 (t) (n + 1)! , n = 1, 2, . . . , t ∈ (x 0 , x n ) if f(x) has a continuous (n + 1)st derivative. Example 3.3 Error estimate of linear interpolation Solution. Given (x 0 , f 0 ) and (x 1 , f 1 ) we set L 0 (x) = x −x 1 x 0 −x 1 , L 1 (x) = x −x 0 x 1 −x 0 , which gives the Lagrange polynomial p 1 (x) = L 0 (x)f 0 + L 1 (x)f 1 = x −x 1 x 0 −x 1 f 0 + x −x 0 x 1 −x 0 f 1 . In the case under consideration, x 0 = 9.0, x 1 = 9.5, f 0 = 2.1972, and f 1 = 2.2513. and ln 9.2 ≈ ˜ a = p 1 (9.2) = L 0 (9.2)f 0 + L 1 (9.2)f 1 = 0.6 · 2.1972 + 0.4 · 2.2513 = 2.2188. The error is = a − ˜ a = 2.2192 −2.2188 = 0.0004. Estimate the error according to the general formula with n = 1 1 (x) = f(x) −p 1 (x) = (x −x 0 )(x −x 1 ) f (t) 2 , t ∈ (9.0, 9.5) 18 with f(t) = ln t, f (t) = 1/t, f (t) = −1/t 2 . Hence 1 (x) = (x −9.0)(x −9.5) (−1) t 2 , 1 (9.2) = (0.2)(−0.3) (−1) 2t 2 = 0.03 t 2 (t ∈ (9.0, 9.5)), 0.00033 = 0.03 9.5 2 = min t∈[9.0,9.5] ¸ ¸ ¸ ¸ 0.03 t 2 ¸ ¸ ¸ ¸ ≤ | 1 (9.2)| ≤ max t∈[9.0,9.5] ¸ ¸ ¸ ¸ 0.03 t 2 ¸ ¸ ¸ ¸ = 0.03 9.0 2 = 0.00037 so that 0.00033 ≤ | 1 (9.2)| ≤ 0.00037, which disagrees with the obtained error = a − ˜ a = 0.0004. In fact, repetition of computations with 5D instead of 4D gives ln 9.2 ≈ ˜ a = p 1 (9.2) = 0.6 · 2.19722 + 0.4 · 2.25129 = 2.21885. with an actual error = 2.21920 −2.21885 = 0.00035 which lies in between 0.00033 and 0.00037. A discrepancy between 0.0004 and 0.00035 is thus caused by the round-off-to-4D error which is not taken into acount in the general formula for the interpolation error. 3.2 Estimation by the error principle First we calculate p 1 (9.2) = 2.21885 and then p 2 (9.2) = 0.54 · 2.1972 + 0.48 · 2.2513 −0.02 · 2.3979 = 2.21916 from Example 2.3 but with 5D. The difference p 2 (9.2) −p 1 (9.2) = 2.21916 −2.21885 = 0.00031 is the approximate error of p 1 (9.2): 0.00031 is an approximation of the error 0.00035 obtained above. The Lagrange interpolation formula (22) is very unconvenient for actual calculation. More- over, when computing with polynomials P n (x) for varying n, the calculation of P n (x) for a particular n is of little use in calculating a value with a larger n. These problems are avoided by using another formula for P n (x), employing the divided differences of the data being interpolated. 3.3 Divided differences Assume that it is given the grid of points x 0 , x 1 , x 2 , . . . ; x i = x j , i = j and corresponding values of a function f(x) : f 0 , f 1 , f 2 , . . . The first divided differences are defined as f[x 0 , x 1 ] = f 1 −f 0 x 1 −x 0 ; f[x 1 , x 2 ] = f 2 −f 1 x 2 −x 1 ; . . . 19 The second divided differences are f[x 0 , x 1 , x 2 ] = f[x 1 , x 2 ] −f[x 0 , x 1 ] x 2 −x 0 ; (23) f[x 1 , x 2 , x 3 ] = f[x 2 , x 3 ] −f[x 1 , x 2 ] x 3 −x 1 ; . . . and of order n f[x 0 , x 1 , . . . , x n , x n+1 ] = f[x 1 , x 2 , . . . , x n+1 ] −f[x 0 , x 1 , . . . x n ] x n+1 −x 0 . It is easy to see that the order of x 0 , x 1 , x 2 , . . . , x n will not make a difference in the calculation of divided difference. In other words, any permutation of the grid points does not change the value of divided difference. Indeed, for n = 1 f[x 1 , x 0 ] = f 0 −f 1 x 0 −x 1 = f 1 −f 0 x 1 −x 0 = f[x 0 , x 1 ]. For n = 2, we obtain f[x 0 , x 1 , x 2 ] = f 0 (x 0 −x 1 )(x 0 −x 2 ) + f 1 (x 1 −x 0 )(x 1 −x 2 ) + f 2 (x 2 −x 0 )(x 2 −x 1 ) . If we interchange values of x 0 , x 1 and x 2 , then the fractions of the right-hand side will inter- change their order, but the sum will remain the same. When the grid points (nodes) are spaced at equal intervals (forming the uniform grid), the divided difference are coupled with the forward differences by simple formulas. Set x j = x 0 + jh, j = 0, 1, 2, . . . and assume that f j = f(x 0 + jh) are given. Then the first forward difference is f[x 0 , x 1 ] = f[x 0 , x 0 + h] = f(x 0 + h) −f(x 0 ) x 0 + h −x 0 = f 1 −f 0 h = ∆f 0 1!h . For the second forward difference we have f[x 0 , x 1 , x 2 ] = 1 2h _ ∆f 1 1!h − ∆f 0 1!y _ = ∆ 2 f 0 2!h 2 (24) and so on. For arbitrary n = 1, 2, . . . f[x 0 , x 0 + h, . . . , x 0 + nh] = ∆ n f 0 n!h n . It is easy to calculate divided differences using the table of divided differences 20 x 0 f(x 0 ) f[x 0 , x 1 ] x 1 f(x 1 ) f[x 0 , x 1 , x 2 ] f[x 1 , x 2 ] f[x 0 , x 1 , x 2 , x 3 ] x 2 f(x 2 ) f[x 1 , x 2 , x 3 ] f[x 2 , x 3 ] f[x 1 , x 2 , x 3 , x 4 ] . . . . . . . . . . . . f[x n−1 , x n ] x n f(x n ) Construct the table of divided differences for the function f(x) = 1 1 + x 2 , at the nodes x k = kh, k = 0, 1, 2, . . . , 10 with the step h = 0.1. The values of f k = f(x k ) are found using the table of the function 1 x with (fixed number of) three decimal places. In the first column we place the values x k , in the second, f k , and in the third, the first divided difference f[x k , x k+1 ] = ∆f k h , etc.: 0.0 1.000 −0.100 0.1 0.990 −0.900 −0.280 0.167 0.2 0.962 −0.850 −0.450 0.3 0.917 Let P n (x) denote the polynomial interpolating f(x) at the nodes x i for i = 0, 1, 2, . . . , n. Thus, in general, the degree of P n (x) is less or equal n and P n (x i ) = f(x i ), i = 0, 1, 2, . . . n. (25) Then P 1 (x) = f 0 + (x − x 0 )f[x 0 , x 1 ]; (26) P 2 (x) = f 0 + (x − x 0 )f[x 0 , x 1 ] + (x − x 0 )(x − x 1 )f[x 0 , x 1 , x 2 ] (27) . . . P n (x) = f 0 + (x − x 0 )f[x 0 , x 1 ] + . . . +(x − x 0 )(x − x 1 )(x − x n−1 )f[x 0 , x 1 , . . . x n ]. (28) This is called the Newton’s divided difference formula for the interpolation polynomial. Note that for k ≥ 0 P k+1 = P k (x) + (x − x 0 ) . . . (x − x k )f[x 0 , x 1 , . . . x k+1 ]. 21 Thus we can go from degree k to degree k +1 with a minimum of calculation, once the divided differences have been computed (for example, with the help of the table of finite differences). We will consider only the proof of (26) and (27). For the first case, one can see that P 1 (x 0 ) = f 0 and P 1 (x 1 ) = f 0 + (x 1 −x 0 ) f(x 1 ) − f(x 0 ) x 1 − x 0 = f 0 + (f 1 − f 0 ) = f 1 . Thus P 1 (x) is a required interpolation polynomial, namely, it is a linear function which satisfies the interpolation conditions (25). For (27) we have the polynomial of a degree ≤ 2 P 2 (x) = P 1 (x) + (x − x 0 )(x − x 1 )f[x 0 , x 1 , x 2 ] and for x 0 , x 1 P 2 (x i ) = P 1 (x i ) + 0 = f i , i = 0, 1. Also, P 2 (x 2 ) = f 0 + (x 2 −x 0 )f[x 0 , x 1 ] + (x 2 −x 0 )(x 2 −x 1 )f[x 0 , x 1 , x 2 ] = f 0 + (x 2 −x 0 )f[x 0 , x 1 ] + (x 2 −x 1 )(f[x 1 , x 2 ] − f[x 0 , x 1 ]) = f 0 + (x 1 −x 0 )f[x 0 , x 1 ] + (x 2 −x 1 )f[x 1 , x 2 ] = f 0 + (f 1 − f 0 ) + (f 2 − f 1 ) = f 2 . By the uniqueness of interpolation polynomial this is the quadratic interpolation polynomial for the function f(x) at x 0 , x 1 , x 2 . In the general case the formula for the interpolation error may be represented as follows f(x) = P n (x) + n (x), x ∈ [x 0 , x n ] where x 0 , x 1 , . . . , x n are (different) interpolation points, P n (x) is interpolation polynomial con- structed by any of the formulas verified above and n (x) is the interpolation error. Example 3.5 Newton’s divided difference interpolation formula Compute f(9.2) from the given values. 8.0 2.079442 0.117783 9.0 2.197225 −0.006433 0.108134 0.000411 9.5 2.251292 −0.005200 0.097735 11.0 2.397895 We have f(x) ≈ p 3 (x) = 2.079442 + 0.117783(x −8.0)− 0.006433(x −8.0)(x −9.0) + 0.000411(x −8.0)(x −9.0)(x −9.5). 22 At x = 9.2, f(9.2) ≈ 2.079442 + 0.141340 −0.001544 −0.000030 = 2.219208. We can see how the accuracy increases from term to term: p 1 (9.2) = 2.220782, p 2 (9.2) = 2.219238, p 3 (9.2) = 2.219208. Note that interpolation makes sense only in the closed interval between the first (minimal) and the last (maximal) interpolation points. Newton’s interpolation formula (28) becomes especially simple when interpolation points x 0 , x 0 + h, x 0 + 2h, . . . are spaced at equal intervals. In this case one can rewrite Newton’s interpolation polynomial using the formulas for divided differences and introducing the variable r = x −x 0 h such that x = x 0 + rh, x −x 0 = rh, (x −x 0 )(x −x 0 −h) = r(r −1)h 2 , . . . Substituting this new variable into (28) we obtain the Newton’s forward difference inter- polation formula f(x) ≈ P n (x) = f 0 + r∆f 0 + r(r −1) 2! ∆ 2 f 0 + . . . + r(r −1) . . . (r −n + 1) n! ∆ n f 0 . Error estimate is given by n (x) = f(x) −p n (x) = h n+1 (n + 1)! r(r −1) . . . (r −n)f n+1 (t), n = 1, 2, . . . , t ∈ (x 0 , x n ) if f(x) has a continuous (n + 1)st derivative. Example 3.6 Newton’s forward difference formula. Error estimation Compute cosh(0.56) from the given values and estimate the error. Solution. Construct the table of forward differences 0.5 1.127626 0.057839 0.6 1.185645 0.011865 0.069704 0.000697 0.7 1.255169 0.012562 0.082266 0.8 1.337435 23 We have x = 0.56, x 0 = 0.50, h = 0.1, r = x −x 0 h = 0.56 −0.50 0.1 = 0.6 and cosh(0.56) ≈ p 3 (0.56) = 1.127626+0.6·0.057839+ 0.6(−0.4) 2 ·0.011865+ 0.6(−0.4)(−1.4) 6 ·0.000697 = 1.127626 + 0.034703 −0.001424 + 0.000039 = 1.160944. Error estimate. We have f(t) = cosh(t) with f (4) (t) = cosh (4) (t) = cosh(t), n = 3, h = 0.1, and r = 0.6, so that 3 (0.56) = cosh(0.56) −p 3 (0.56) = (0.1) 4 (4)! 0.6(0.6 −1)(0.6 −2)(0.6 −3) cosh (4) (t) = Acosh(t), t ∈ (0.5, 0.8) where A = −0.0000036 and Acosh 0.8 ≤ 3 (0.56) ≤ Acosh 0.5 so that p 3 (0.56) + Acosh 0.8 ≤ cosh(0.56) ≤ p 3 (0.56) + Acosh 0.5. Numerical values 1.160939 ≤ cosh(0.56) ≤ 1.160941 Problem 3.1 (see Problem 17.3.1, AEM) Compute ln 9.3 from ln 9.0 = 2.1972 and ln 9.5 = 2.2513 by the linear Lagrange interpolation and determine the error from a = ln 9.3 = 2.2300 (exact to 4D). Solution. Given (x 0 , f 0 ) and (x 1 , f 1 ) we set L 0 (x) = x −x 1 x 0 −x 1 , L 1 (x) = x −x 0 x 1 −x 0 , which gives the Lagrange polynomial p 1 (x) = L 0 (x)f 0 + L 1 (x)f 1 = x −x 1 x 0 −x 1 f 0 + x −x 0 x 1 −x 0 f 1 . In the case under consideration, x 0 = 9.0, x 1 = 9.5, f 0 = 2.1972, and f 1 = 2.2513. L 0 (x) = x −9.5 (−0.5) = 2(9.5 −x) = 19 −2x, L 1 (x) = x −9.0 0.5 = 2(x −9) = 2x −18. The Lagrange polynomial is p 1 (x) = L 0 (x)f 0 + L 1 (x)f 1 = (19−2x)2.1972+(2x−18)2.2513 = 2x(2.2513−2.1972)+19·2.1972−18·2.2513 = 0.1082x+1.2234. Now calculate L 0 (9.3) = 9.3 −9.5 9.0 −9.5 = 0.4, L 1 (9.3) = 9.3 −9.0 9.5 −9.0 = 0.6, 24 and get the answer ln 9.3 ≈ ˜ a = p 1 (9.3) = L 0 (9.3)f 0 + L 1 (9.3)f 1 = 0.4 · 2.1972 + 0.6 · 2.2513 = 2.2297. The error is = a − ˜ a = 2.2300 −2.2297 = 0.0003. Problem 3.2 (see Problem 17.3.2, AEM) Estimate the error of calculating ln 9.3 from ln 9.0 = 2.1972 and ln 9.5 = 2.2513 by the linear Lagrange interpolation (ln 9.3 = 2.2300 exact to 4D). Solution. We estimate the error according to the general formula with n = 1 1 (x) = f(x) −p 1 (x) = (x −x 0 )(x −x 1 ) f (t) 2 , t ∈ (9.0, 9.5) with f(t) = ln t, f (t) = 1/t, f (t) = −1/t 2 . Hence 1 (x) = (x −9.0)(x −9.5) (−1) t 2 , 1 (9.3) = (0.3)(−0.2) (−1) 2t 2 = 0.03 t 2 (t ∈ (9.0, 9.5)), 0.00033 = 0.03 9.5 2 = min t∈[9.0,9.5] ¸ ¸ ¸ ¸ 0.03 t 2 ¸ ¸ ¸ ¸ ≤ | 1 (9.3)| ≤ max t∈[9.0,9.5] ¸ ¸ ¸ ¸ 0.03 t 2 ¸ ¸ ¸ ¸ = 0.03 9.0 2 = 0.00037 so that 0.00033 ≤ | 1 (9.3)| ≤ 0.00037, which disagrees with the obtained error = a−˜ a = 0.0003 because in the 4D computations we cannot round-off the last digit 3. In fact, repetition of computations with 5D instead of 4D gives ln 9.3 ≈ ˜ a = p 1 (9.3) = 0.4 · 2.19722 + 0.6 · 2.25129 = 0.87889 + 1.35077 = 2.22966 with an actual error = 2.23001 −2.22966 = 0.00035 which lies between 0.00033 and 0.00037. A discrepancy between 0.0003 and 0.00035 is thus caused by the round-off-to-4D error which is not taken into acount in the general formula for the interpolation error. Problem 3.3 (optional; see Problem 17.3.3, AEM) Compute e −0.25 and e −0.75 by linear interpolation with x 0 = 0, x 1 = 0.5 and x 0 = 0.5, x 1 = 1. Then find p 2 (x) interpolating e −x with x 0 = 0, x 1 = 0.5, and x 2 = 1 and from it e −0.25 and e −0.75 . Compare the errors of these linear and quadratic interpolation. Solution. Given (x 0 , f 0 ) and (x 1 , f 1 ) we set L 0 (x) = x −x 1 x 0 −x 1 , L 1 (x) = x −x 0 x 1 −x 0 , which gives the Lagrange polynomial p 1 (x) = L 0 (x)f 0 + L 1 (x)f 1 = x −x 1 x 0 −x 1 f 0 + x −x 0 x 1 −x 0 f 1 . 25 In the case of linear interpolation, we will interpolate e x and take first the nodes x 0 = −0.5 and x 1 = 0 and f 0 = e −0.5 = 0.6065 and f 1 = e 0 = 1.0000. L 0 (x) = x −0 (−0.5) = −2x, L 1 (x) = x + 0.5 (0.5) = 2(x + 0.5) = 2x + 1. The Lagrange polynomial is p 1 (x) = L 0 (x)f 0 + L 1 (x)f 1 = −2x · 0.6065 + (2x + 1)1.0000 = 2x(1.0000 −0.6065) + 1 = 2 · 0.3935x + 1. The answer e −0.25 ≈ p 1 (−0.25) = −0.25 · 2 · 0.3935 + 1 = 1 −0.1967 = 0.8033. The error is = e −0.25 −p 1 (−0.25) = 0.7788 −0.8033 = −0.0245. Now take the nodes x 0 = −1 and x 1 = −0.5 and f 0 = e −1 = 0.3679 and f 1 = e −0.5 = 0.6065. L 0 (x) = x + 0.5 (−0.5) = −2(x + 0.5) = −2x −1, L 1 (x) = x + 1 (0.5) = 2(x + 1). The Lagrange polynomial is p 1 (x) = L 0 (x)f 0 + L 1 (x)f 1 = (−2x−1) · 0.3679+(2x+2)0.6065 = 2x(0.6065−0.3679) −0.3679+2· 0.6065 = 2· 0.2386x+0.8451. The answer e −0.75 ≈ p 1 (−0.75) = −0.75 · 2 · 0.2386 + 0.8451 = −0.3579 + 0.8451 = 0.4872. The error is = e −0.75 −p 1 (−0.75) = 0.4724 −0.4872 = −0.0148. In the case of quadratic interpolation, we will again interpolate e x and take the nodes x 0 = −1, x 1 = −0.5, and x 2 = 0 and f 0 = e −1 = 0.3679, f 1 = e −0.5 = 0.6065 and f 2 = 1.0000. Given (x 0 , f 0 ), (x 1 , f 1 ), and (x 2 , f 2 ) we set L 0 (x) = l 0 (x) l 0 (x 0 ) = (x −x 1 )(x −x 2 ) (x 0 −x 1 )(x 0 −x 2 ) , L 1 (x) = l 1 (x) l 1 (x 1 ) = (x −x 0 )(x −x 2 ) (x 1 −x 0 )(x 1 −x 2 ) , L 2 (x) = l 2 (x) l 2 (x 2 ) = (x −x 0 )(x −x 1 ) (x 2 −x 0 )(x 2 −x 1 ) , which gives the quadratic Lagrange polynomial p 2 (x) = L 0 (x)f 0 + L 1 (x)f 1 + L 2 (x)f 2 . In the case under consideration, calculate L 0 (x) = (x + 0.5)(x) (−0.5)(−1) = 2x(x + 0.5); L 0 (−0.25) = −0.5 · 0.25 = −0.125, L 0 (−0.75) = (−1.5) · (−0.25) = 0.375. L 1 (x) = (x + 1)(x) (0.5)(−0.5) = −4x(x + 1); L 1 (−0.25) = 1 · 0.75 = 0.75, L 1 (−0.75) = 3 · 0.25 = 0.75. 26 L 2 (x) = (x + 1)(x + 0.5) (1)(0.5) = 2(x + 0.5)(x + 1); L 2 (−0.25) = 0.5 · 0.75 = 0.375, L 2 (−0.75) = (−0.5) · 0.25 = −0.125. The answers are as follows: e −0.25 ≈ p 2 (−0.25) = L 0 (−0.25)f 0 + L 1 (−0.25)f 1 + L 2 (−0.25)f 2 = −0.1250 · 0.3679 + 0.7500 · 0.6065 + 0.3750 · 1.0000 = −0.0460 + 0.4549 + 0.3750 = 0.7839. The error is = e −0.25 −p 2 (−0.25) = 0.7788 −0.7839 = −0.0051.. e −0.75 ≈ p 2 (−0.75) = L 0 (−0.75)f 0 + L 1 (−0.75)f 1 + L 2 (−0.75)f 2 = 0.3750 · 0.3679 + 0.7500 · 0.6065 −0.1250 · 1.0000 = 0.1380 + 0.4549 −0.1250 = 0.4679. The error is = e −0.75 −p 2 (−0.75) = 0.4724 −0.4679 = 0.0045.. The quadratic Lagrange polynomial is p 2 (x) = L 0 (x)f 0 + L 1 (x)f 1 + L 2 (x)f 2 = 2x(x + 0.5) · 0.3679 −4x(x + 1) · 0.6065 + 2(x + 0.5)(x + 1) · 1.0000 = 0.3095x 2 −0.9418x + 1. Problem 3.4 (quadratic interpolation) (see Problem 17.3.5, AEM) Calculate the Lagrange polynomial p 2 (x) for 4-D values of the Gamma-function Γ(x) = _ ∞ 0 e −t t x−1 dt, Γ(1.00) = 1.0000, Γ(1.02) = 0.9888, and Γ(1.04) = 0.9784, and from it the approximation of Γ(1.01) and Γ(1.03). Solution. We will interpolate Γ(x) taking the nodes x 0 = 1.00, x 1 = 1.02, and x 2 = 1.04 and f 0 = 1.0000, f 1 = 0.9888, and f 2 = 0.9784. Given (x 0 , f 0 ), (x 1 , f 1 ), and (x 2 , f 2 ) we set L 0 (x) = l 0 (x) l 0 (x 0 ) = (x −x 1 )(x −x 2 ) (x 0 −x 1 )(x 0 −x 2 ) , L 1 (x) = l 1 (x) l 1 (x 1 ) = (x −x 0 )(x −x 2 ) (x 1 −x 0 )(x 1 −x 2 ) , L 2 (x) = l 2 (x) l 2 (x 2 ) = (x −x 0 )(x −x 1 ) (x 2 −x 0 )(x 2 −x 1 ) , which gives the quadratic Lagrange polynomial p 2 (x) = L 0 (x)f 0 + L 1 (x)f 1 + L 2 (x)f 2 . In the case under consideration, calculate L 0 (x) = (x −1.02)(x −1.04) (−0.02)(−0.04) = 1250(x −1.02)(x −1.04); 27 L 0 (1.01) = (−0.01)(−0.03) (−0.02)(−0.04) = 3 8 = 0.375, L 0 (1.03) = (0.01)(−0.01) (−0.02)(−0.04) = − 1 8 = −0.125. L 1 (x) = (x −1)(x −1.04) (0.02)(−0.02) = −2500(x −1)(x −1.04); L 1 (1.01) = (0.01)(−0.03) (0.02)(−0.02) = 3 4 = 0.75, L 1 (1.03) = (0.03)(−0.01) (0.02)(−0.02) = 3 4 = 0.75. L 2 (x) = (x −1)(x −1.02) (0.04)(0.02) = 1250(x −1)(x −1.02); L 2 (1.01) = (0.01)(−0.01) (0.04)(0.02) = − 1 8 = −0.125, L 1 (1.03) = (0.03)(0.01) (0.04)(0.02) = 3 8 = 0.375. The answers are as follows: Γ(1.01) ≈ p 2 (1.01) = L 0 (1.01)f 0 + L 1 (1.01)f 1 + L 2 (1.01)f 2 = 0.3750 · 1.0000 + 0.7500 · 0.9888 −0.1250 · 0.9784 = 0.3750 + 0.7416 −0.1223 = 0.9943. The error is = Γ(1.01) −p 2 (1.01) = 0.9943 −0.9943 = 0.0000. The result is exact to 4D. Γ(1.03) ≈ p 2 (1.03) = L 0 (1.03)f 0 + L 1 (1.03)f 1 + L 2 (1.03)f 2 = −0.1250 · 1.0000 + 0.7500 · 0.9888 + 0.3750 · 0.9784 = −0.1250 + 0.7416 + 0.3669 = 0.9835. The error is = Γ(1.03) −p 2 (1.03) = 0.9835 −0.9835 = 0.0000. The result is exact to 4D. The quadratic Lagrange polynomial is p 2 (x) = L 0 (x)f 0 + L 1 (x)f 1 + L 2 (x)f 2 = 1250(x−1.02)(x−1.04)·1.0000−2500(x−1)(x−1.04)·0.9888+1250(x−1)(x−1.02)·0.9784 = = x 2 (1250(·1.9784 −2 · 0.9888)) + . . . = x 2 (1250((2 −0.0216) −2(1 −·0.0112))) + . . . = x 2 (1250(−0.0216+2·0.0112))+. . . = x 2 (1250·0.0008)+. . . = x 2 ·1.000+. . . = x 2 −2.580x+2.580. Problem 3.5 (Newton’s forward difference formula) (see Problem 17.3.11, AEM) Compute Γ(1.01), Γ(1.03), and Γ(1.05) by Newton’s forward difference formula. Solution. Construct the table of forward differences 1.00 1.0000 0.0112 1.02 0.9888 0.0008 0.0104 1.04 0.9784 From Newton’s forward difference formula, we have x = 1.01, 1.03, 1.05; x 0 = 1.00, h = 0.02, r = x −1 h = 50(x −1); p 2 (x) = f 0 + r∆f 0 + r(r −1) 2 ∆ 2 f 0 = 1.000 −0.0112r + 0.0008 r(r −1) 2 = x 2 −2.580x + 2.580, 28 which coincides with the quadratic interpolation polynomial derived above. Therefore, we can perform direct calculations to obtain Γ(1.01) ≈ p 2 (1.01) = 0.9943, Γ(1.03) ≈ p 2 (1.03) = 0.9835, Γ(1.05) ≈ p 2 (1.05) = 0.9735. Problem 3.6 (lower degree) (see Problem 17.3.13, AEM) What is the degree of the interpolation polynomial for the data (1,5), (2,18), (3,37), (4,62), (5,93)? Solution. We find the polynomial proceeding from an assumption that it has the lowest possible degree n = 2 and has the form p 2 (x) = ax 2 + bx + c. In fact, it is easy to check that the required polynomial is not a linear function, Ax + B, because A + B = 5 2A + B = 18 yields A = 13 and B = −8, and 3A + B = 37 4A + B = 62 yields A = 25 and B = −38. For the determination of the coefficients a, b, c we have the system of three linear algebraic equations a + b + c = 5, 4a + 2b + c = 18 9a + 3b + c = 37 Subtracting the first equation from the second and from the third, we obtain 3a + b = 13 8a + 2b = 32 or 6a + 2b = 26 8a + 2b = 32 which yields 2a = 6, a = 3, b = 13 −3a = 4, and c = 5 −b −a = −2. Thus, the desired polynomial of degree n = 2 is p 2 (x) = 3x 2 + 4x −2. It is easy to see that p 2 (1) = 5, p 2 (2) = 18, p 2 (3) = 37, p 2 (4) = 62, and p 2 (5) = 93. 4 Numerical Integration and Differentiation Numerical integration means numerical evaluation of integrals J = _ b a f(x)dx, where a abd b are given numbers and f(x) is a function given analytically by a formula or empirically by a table of values. 29 4.1 Rectangular rule. Trapezoidal rule Subdivide the interval of integration a ≤ x ≤ b into n subintervals of equal length h = (b−a)/n and in each subinterval approximate f(x) by the constant f(x ∗ j ), the value of f(x) at the mid- point x ∗ j of the jth subinterval. The f(x) is approximated by a step function (piecewise constant function). The n rectangles have the areas f(x ∗ 1 ), f(x ∗ 2 ), . . . , f(x ∗ n ), and the rectangular rule is J = _ b a f(x)dx ≈ h[f(x ∗ 1 ) + f(x ∗ 2 ) + . . . + f(x ∗ n )], h = (b −a)/n. (29) The trapezoidal rule is generally more accurate. Take the same subdivision of a ≤ x ≤ b into n subintervals of equal length and approximate f(x) by a broken line of segments (chords) with endpoints [a, f(a)], [x 1 , f(x 1 )], [x 2 , f(x 2 )], . . . , [x n , f(x n )] on the graph of f(x). The the area under the graph of f(x) is approximated by n trapezoids of areas 1 2 [f(a) + f(x 1 )], 1 2 [f(x 1 ) + f(x 2 )], . . . , 1 2 [f(x n−1 ) + f(b)]. Calculating the sum we obtain the trapezoidal rule J = _ b a f(x)dx ≈ h[ 1 2 f(a) + f(x 1 ) + f(x 2 ) + . . . + f(x n−1 ) + 1 2 f(b)], h = (b −a)/n. (30) 4.2 Error bounds and estimate for the trapezoidal rule Recall that the error n (x) of the interpolation by the Lagrange polymonial p n (x) of degree n is given by n (x) = f(x) −p n (x) = (x −x 0 )(x −x 1 ) . . . (x −x n ) f n+1 (t) (n + 1)! , n = 1, 2, . . . , t ∈ (x 0 , x n ) if f(x) has a continuous (n + 1)st derivative. The error estimate of the linear interpolation is, respectively, (for n = 1) is 1 (x) = f(x) −p 1 (x) = (x −x 0 )(x −x 1 ) f (t) 2 , t ∈ [a, b] (31) We will use this formula to determine the error bounds and estimate for the trapezoidal rule: integrate it over x from a = x 0 to x 1 = x 0 + h to obtain J = _ x 0 +h x 0 f(x)dx − h 2 [f(x 0 ) + f(x 1 )] = _ x 0 +h x 0 (x −x 0 )(x −x 0 −h) f (t(x)) 2 dx. Setting x−x 0 = v and applying the mean value theorem, which we can use because (x−x 0 )(x− x 0 −h) does not change sign, we find that the right-hand side equals _ h 0 (v)(v −h)dv · f ( ˜ t) 2 = − h 3 12 f ( ˜ t), ˜ t ∈ [x 0 , x 1 ]. (32) which gives the formula for the local error of the trapezoidal rule. 30 Find the total error of the trapezoidal rule with any n. We have h = (b −a)/n, nh 3 = n(b −a) 3 /n 3 , (b −a) 2 = n 2 h 2 ; therefore = _ b a f(x)dx −h[ 1 2 f(a) + f(x 1 ) + f(x 2 ) + . . . + f(x n−1 ) + 1 2 f(b)] = − (b −a) 3 12n 2 f (τ) = −h 2 (b −a) 12 f (τ), τ ∈ [a, b]. Error bounds for the trapezoidal rule KM 2 ≤ ≤ KM ∗ 2 , where K = − (b −a) 3 12n 2 = −h 2 (b −a) 12 and M 2 = max x∈[a,b] f (x), M ∗ 2 = min x∈[a,b] f (x). Example 4.1 The error for the trapezoidal rule Estimate the error of evaluation of J = _ 1 0 e −x 2 dx ≈ 0.746211 (n = 10) by the trapezoidal rule. Solution. We have (e −x 2 ) = 2(2x 2 −1))e −x 2 , (e −x 2 ) = 8xe −x 2 (3/2 −x 2 ) > 0, 0 < x < 1, Therefore, max x∈[0,1] f (x) = f (1) = M 2 = 0.735759, min x∈[0,1] f (x) = f (0) = M ∗ 2 = −2. Furthermore, K = − (b −a) 3 12n 2 = − 1 12 · 10 2 = − 1 1200 and KM 2 = −0.000614, KM ∗ 2 = 0.001667, so that the error −0.000614 ≤ ≤ 0.001667, (33) and the estimation is 0.746211 −0.000614 = 0.745597 ≤ J ≤ 0.746211 + 0.001667 = 0.747878. (34) The 6D-exact value is J = 0.746824 : 0.745597 < 0.746824 < 0.747878. 31 4.3 Numerical integration using MATLAB The MATLAB command trapz(x,y) computes the integral of y as a function of of x. Vectors x and y have the same length and (x i , y i ) represents a point on the curve. Example 4.2 Evaluate J = _ 1 0 e −x 2 dx by the trapezoidal rule. Using trapz command we first create vector x. We try this vector with 5 and 10 values: x5 = linspace(0,1,5); x10 = linspace(0,1,10); Then we create vector y as a function of x: y5 = exp(–x5.∗x5); y10 = exp(–x10.∗x10); Now the integral can be computed: integral5 = trapz(x5,y5), integral10 = trapz(x10,y10) returns integral5 = 0.74298409780038 integral10 = 0.74606686791267 4.4 Numerical differentiation Since, by definition of the derivative of a function f(x) at a point x, f (x) = lim h→0 f(x + h) −f(x) h , we may write the approximate formulas for the first and second derivatives f (x) ≈ ∆f 1 = f 1 −f 0 h , f (x) ≈ ∆f 2 −∆f 1 h = f 2 −2f 1 + f 0 h 2 , etc., where f 0 = f(x) and f 1 = f(x + h). 32 If we have an equispaced grid {x i } n i=1 with the spacing h = x i+1 − x i (i = 1, 2, . . . , n − 1, n ≥ 2) then the corresponding approximate formulas for the first and second derivatives written at each node x i = x 1 + (i −1)h (i = 1, 2, . . . , n + 1) are f (x i ) ≈ f i+1 −f i h , i = 1, 2, . . . , n −1, f (x i ) ≈ f i+1 −2f i + f i−1 h 2 , i = 2, 3, . . . , n −1. More accurate formulas are f (x) ≈ p 2 (x) = 2x −x 1 −x 2 2h 2 f 0 − 2x −x 0 −x 2 h 2 f + 2x −x 0 −x 1 2h 2 f 2 . Evaluating these ratios at x 0 , x 1 , x 2 we obtain f 0 ≈ 4f 1 −3f 0 −f 2 2h , f 1 ≈ f 2 −f 0 2h , f 2 ≈ −4f 1 + f 0 + 3f 2 2h . 5 Approximation of Functions Let us introduce the basic terms: – norm; uniform norm; L 2 -norm; – polynomial of best approximation; – to alternate; alternating; – reference; – reference set; reference deviation; reference-polynomial; – level; levelled reference-polynomial; – to minimize; – to terminate; So far we have primarly been concerned with representing a given function f(x) by an interpolating polynomial P n (x) of the degree n (which coincides with the given values of f(x) at the interpolation points). Now we consider approximation of a function f(x) which is known to be continuous in an interval [a, b] (we will denote it f(x) ∈ C[a, b]) by a polynomial P n (x) of degree n or less which does not necessarily coincide with f(x) at any points. This approximation polynomial should be “close enough” for x ∈ [a, b] to the given function. There are various measures of the closeness of the approximation of f(x) by P n (x). We shall be interested in finding a polynomial P n (x) which minimizes the quantity max a≤x≤b |P n (x) − f(x)|, 33 which is called the uniform norm of P n − f and is denoted by ||P n −f|| or ||P n −f|| C . Another measure which is often used is the L 2 -norm given by ||P n − f|| 2 = { _ b a (P n (x) −f(x)) 2 dx} 1 2 . We now state without proof two mathematical theorems which are related to the approxi- mation of functions in the uniform norm. Theorem 5.1. If f(x) ∈ C[a, b], then given any > 0 there exists a polynomial P n (x) such that for x ∈ [a, b] |P n (x) − f(x)| ≤ . This is the Weierstrass theorem. Theorem 5.2. If f(x) ∈ C[a, b], for a given integer n there is a unique polynomial π n (x) of degree n or less such that δ n = max a≤x≤b |π n (x) − f(x)| ≤ max a≤x≤b |P n (x) − f(x)| for any polynomial P n (x) of degree n or less. Moreover, there exists a set of n + 2 points x 0 , x 1 , . . . , x n , x n+1 in [a, b], x 0 < x 1 < . . . < x n < x n+1 , such that either π n (x i ) − f(x i ) = (−1) i δ n , i = 0, 1, . . . , n or π n (x i ) − f(x i ) = (−1) i+1 δ n , i = 0, 1, . . . , n. Note that by Theorem 5.1 it follows that δ n →0 as n →∞. No general finite algorithm is known for finding the polynomial π n (x) of best approximation. However, if we replace the interval [a, b] by a discrete set of points, a grid J = ω n+2 , then in a finite number of steps we can find a polynomial π n (x) of degree n or less such that max x∈J |π n (x) − f(x)| ≤ max x∈J |P n (x) − f(x)| for any polynomial P n (x) of degree n or less. We now describe an algorithm for finding π n (x) which is based on the following statement. Given n +2 distinct points x 0 , x 1 , . . . , x n , x n+1 , x 0 < x 1 < . . . < x n < x n+1 , there exist non- vanishing constants λ 0 , λ 1 , . . . , λ n , λ n+1 of alternating sign in the sense that λ 0 λ 1 , λ 1 λ 2 , . . . , λ n λ n+1 are negative and such that for any polynomial P n (x) of degree n or less we have n+1 i=0 λ i P n (x i ) = 0. (35) Moreover, λ 0 = 1 x 0 −x 1 1 x 0 −x 2 . . . 1 x 0 −x n+1 ; λ i = 1 x i −x 0 . . . 1 x i −x i−1 1 x i −x i+1 . . . 1 x i −x n+1 , i = 1, 2, . . . n, (36) λ n+1 = 1 x n+1 −x 0 1 x n+1 −x 1 . . . 1 x n+1 −x n . 34 Let us explain the proof considering the simplest case n = 1 when one has three points x 0 , x 1 , x 2 , three non-zero constants λ 0 = 1 x 0 −x 1 1 x 0 −x 2 , λ 1 = 1 x 1 −x 0 1 x 1 −x 2 , (37) λ 2 = 1 x 2 −x 0 1 x 2 −x 1 , and P 1 (x) = ax+b is a polynomial of degree 1 or less. Application of formula (35) means that one interpolates a linear function P 1 (x) by interpolation polynomial of degree 1 or less using interpolation points x 0 , x 1 , hence (35) yields the identities P 1 (x) = ax + b = 1 i=0 { 1 j=i x −x j x i −x j }P 1 (x i ) = x −x 1 x 0 −x 1 (ax 0 + b) + x −x 0 x 1 −x 0 (ax 1 + b) ≡ ax + b. If we substitute x = x 2 into the last identity and divide by (x 2 −x 0 )(x 2 −x 1 ) we obtain 1 x 0 −x 1 1 x 0 −x 2 P 1 (x 0 ) + 1 x 1 −x 0 1 x 1 −x 2 P 1 (x 1 ) + 1 x 2 −x 0 1 x 2 −x 1 P 1 (x 2 ) = 0. what coincides with the required equality for the considered case. Let us introduce the following terminology. 1. A reference set is a set (grid) J = ω n+2 of n+2 distinct numbers x 0 , x 1 , . . . , x n , x n+1 , x 0 < x 1 < . . . < x n < x n+1 . 2. A reference-polynomial with respect to the function f(x) defined on J (grid function) and with respect to this reference set is a polynomial P n (x) of degree n or less such that the h i have alternating signs, where h i = P n (x i ) − f(x i ) i = 0, 1, . . . , n, n + 1. 3. A levelled reference-polynomial is a reference-polynomial such that all h i have the same magnitude. Below, we will show that for a given reference set and for a given function f(x) there is a unique levelled reference-polynomial. 4. The reference deviation of a reference set is the value H = |h 0 | = |h 1 | = . . . = |h n+1 | for the levelled-reference polynomial. The next statement proves the existence and uniqueness of a levelled-reference polynomial corresponding to a given reference set x 0 , x 1 , . . . , x n+1 . 35 Theorem 5.3. Given a function f(x) and a reference set R : x 0 , x 1 , . . . , x n+1 , x 0 < x 1 < . . . < x n+1 there exists a unique levelled-reference polynomial P n (x) of degree n or less. More- over, we have P n (x i ) = f(x i ) + (−1) i δ, i = 0, 1, . . . , n + 1, where δ = −sgnλ 0 n+1 i=0 λ i f(x i ) n+1 i=0 |λ i | (38) and where the λ i are given by (36). The reference deviation is given by H = |δ| = | n+1 i=0 λ i f(x i )| n+1 i=0 |λ i | . If Q n (x) is any polynomial of degree n or less such that max x∈R |Q n (x) − f(x)| = H, then Q n (x) ≡ P n (x). Suppose now that x 0 , x 1 , . . . , x n+1 ; x 0 < x 1 < . . . < x n+1 is a reference set and that P n (x) is a reference-polynomial, not necessarily a levelled reference-polynomial with respect to f(x). Let h i = P n (x i ) −f(x i ), i = 0, 1, 2, . . . , n + 1. Then one can show that the reference deviation H is represented in the form H = n+1 i=0 w i |h i |, where the weights w i are positive and are given by w i = |λ i | n+1 i=0 |λ i | , and where the λ i are given by (35). Now we can define the following computational procedures for constructing the polynomial π n (x) of best approximation. 1. Choose any n + 2 points x 0 , x 1 , . . . , x n+1 in J such that x 0 < x 1 < . . . < x n+1 , i.e., choose any reference set, R (R forms a (nonuniform or uniform) grid ω n+2 of n + 2 different points). 2. Determine the levelled reference-polynomial P n (x) and the reference deviation H for R. 3. Find x = z in J such that |P n (z) − f(z)| = max x∈J |P n (x) − f(x)| = ∆. 36 If ∆ = H, terminate the process. 4. Find a new reference set which contains z and is such that P n (x) is a reference-polynomial with respect to the new reference set. This can be done as follows: if z lies in an interval x i ≤ x ≤ x i+1 , i = 0, 1, . . . , n, replace x i by z if P n (z) − f(z) has the same sign as P n (x i ) − f(x i ). Otherwise replace x i+1 by z. If z < x 0 then replace x 0 by z if P n (x 0 ) −f(x 0 ) has the same sign as P n (z) −f(z). Otherwise replace x n+1 by z. A similar procedure is used if z > x n+1 . 5. With the new reference set, return to step 2. The reference deviations are strictly increasing unless, at a given stage, the process termi- nates. For the reference deviation H of the new reference set R , found from R by replacing one value of x by another for which the absolute deviation is strictly greater, is greater than H. Since there are only a finite number of reference sets and since no reference set can occur more than once, the process must eventually terminate. When the process terminates, we have a reference set R ∗ with a reference deviation H ∗ and a polynomial π n (x) which is a levelled reference-polynomial for R ∗ . If τ n (x) is any polynomial of degree n or less, then by Theorem 5.3 either τ n (x) ≡ π n (x) or else τ n (x) has a maximum absolute deviation greater than H ∗ on R ∗ , and hence on J. Thus π n (x) is the polynomial of best approximation. 5.1 Best approximation by polynomials Consider two problems of finding polynomials P 1 (x) of degree 1 or less which give the best approximation of given continuous functions f(x) in the interval [0, 1]. Example 5.1 The function to be approximated is f(x) = x 2 . We will solve this problem analytically by taking P 1 (x) = ax + b. In line with Theorem 5.2 one must find the polynomial π 1 (x) of best approximation by mini- mizing the quantity r 1 (a, b) = max 0≤x≤1 |R 1 (x)|, R 1 (x) = x 2 −ax −b with respect, in general, to all real numbers a, b. We will not consider such a problem in its general statement and impose strong but natural restrictions on the set of admissible values of a and b which obviously follow from the properties of function R 1 (x). Namely, we will assume that a = 1, −0.25 < b < 0 so that all the lines y = P 1 (x) = x + b may lie inside the region ABCD where AD coincides with the tangent line x −0.25 to the curve y = x 2 . Thus in what follows we will take r 1 = r 1 (b), −0.25 < b < 0. It is easy to see that r 1 (b) = max 0≤x≤1 |x 2 −x −b| = max{|R 1 (0)|; |R ( 0.5)|; |R 1 (1)|} = max{−b; |b −0.25|}, 37 since x = 0, 0.5, 1 are the only possible extremal points of the function |R 1 (x)| in the interval [0, 1] and R 1 (0) = R 1 (1) = −b > 0. r 1 (b) is a piecewise linear function. It is clear that min −0.25<b<0 r 1 (b) = r 1 (b 0 ) = 0.125; b 0 = −0.125. Hence the desired polynomial of best approximation π 1 (x) = x −0.125, and ||π 1 (x) −f(x)|| = max 0≤x≤1 |x 2 −x + 0.125| = 0.125. The conditions π 1 (x i ) −f(x i ) = (−1) i+1 δ 2 , i = 0, 1, 2; δ 2 = 0.125 hold at the reference set of points x 0 = 0, x 1 = 0.5, x 2 = 1. The same result may be obtained in the following way. We look for constants a and b such that max 0≤x≤1 |ax + b −x 2 | is minimized. The extreme values of the function ax+b−x 2 occur at x = 0, x = 1 and x = 0.5a (because (ax +b −x 2 ) = a −2x). We will assume that a > 0 since the point x = 0.5a belongs to the interval [0, 1] if 0 < a < 2. From Theorem 5.2 we seek to choose a and b such that ax + b −x 2 = b = −δ 2 (x = 0); ax + b −x 2 = 0.25a 2 + b = δ 2 (x = 0.25a); ax + b −x 2 = a + b −1 = −δ 2 (x = 1); which gives a = 1, b = −0.125, δ 2 = 0.125, and the required polynomial of best approximation π 1 (x) = x −0.125. Example 5.2 Let us now construct the polynomial of best approximation for the function f(x) = e x in the interval [0, 1]. As we shall see below, this problem can be also solved analytically and to the accuracy indicated the polynomial of best approximation is π 1 (x) = 1.718282x + 0.894607 38 and max 0≤x≤1 |π 1 (x) −f(x)| = 0.105933. Let J = ω 101 be the set of points (the grid) t i = i 100 , i = 0, 1, 2, . . . , 100, and let us start with the reference set x 0 = 0, x 1 = 0.5, x 2 = 1.0. From a table of e x we have f(x 0 ) = 1.0000, f(x 1 ) = 1.6487, f(x 2 ) = 2.7183. Let us determine the levelled reference-polynomial P(x) and the reference deviation H. Taking formulas (37) we have λ 0 = 1 x 0 −x 1 1 x 0 −x 2 = 2, λ 1 = 1 x 1 −x 0 1 x 1 −x 2 = −4, λ 2 = 1 x 2 −x 0 1 x 2 −x 1 = 2. We then obtain δ from formula (38): δ = − 2(1.0000) −4(1.6487) + 2(2.7183) 2 + 4 + 2 = −0.1052. Then the reference deviation is H = 0.1052. Suppose that instead of starting with the reference set J we had started with some other reference set and that for this reference set we had the levelled reference-polynomial Q(x) = 0.9 + 1.6x. Since h 0 = Q(x 0 ) −f(x 0 ) = 0.9 −1.0000 = −0.1000, h 1 = Q(x 1 ) −f(x 1 ) = 1.7 −1.6487 = 0.0513, h 2 = Q(x 2 ) −f(x 2 ) = 2.5 −2.7183 = −0.2183 it follows that Q(x) is a reference-levelled polynomial for the reference set J. We now verify (37). We have w 1 = 1 4 , w 2 = 1 2 , w 3 = 1 4 39 and the deviation H = 0.1052 = 1 4 |h 0 | + 1 2 |h 1 | + 1 4 |h 2 | = 1 4 (0.1000) + 1 2 (0.0513) + 1 4 (0.2183). We now determine P(x) by linear interpolation at points x 0 and x 1 with P(x 0 ) = f(x 0 ) + δ = 0.8948, P(x 1 ) = f(x 1 ) −δ = 1.7539. Obviously, P(x) = P(x 0 ) + x −x 0 x 1 −x 0 (P(x 1 ) −P(x 0 )) = 0.8948 + 1.7182x, and P(x 2 ) = 2.6130 = f(x 2 ) −0.1053 = f(x 2 ) + δ. We verify that λ 0 P(x 0 ) + λ 1 P(x 1 ) + λ 2 P(x 2 ) = 2(0.8948) −4(1.7539) + 2(2.6130) = 0. Since the P(x i ) − f(x i ), i = 0, 1, 2 have alternating signs and equal magnitudes, it follows that P(x) is the levelled reference-polynomial for the corresponding reference set. By the direct computation it is found that max x∈J |P(x) −f(x)| = |P(z) −f(z)| = 0.106656, z = 0.54. Since P(z) − f(z) = 0.106656 we replace the reference set {0, 0.5, 1} by {0, 0.54, 1}. We then determine the levelled reference-polynomial π 1 (x) = 1.718282x + 0.894607 for this reference set. Since the deviation is H = 0.105933 and since max x∈J |π 1 (x) −f(x)| = |π 1 (1) −f(1)| = 0.105933, the process is terminated and π 1 (x) is our desired polynomial of best approximation. Let us discuss the choice of the initial reference set. One procedure is simply to let x 0 = a, x n+1 = b and to choose the remaining n points uniformly (forming an uniform grid ω n+2 ), or nearly uniformly, spaced in the interval [a, b]. Another procedure suggested by Chebyshev is to let x i = b + a 2 + b −a 2 cos πi n + 1 , i = 0, 1, . . . , n + 1. In the case a = −1, b = 1, this choice corresponds to the extreme values of the Chebyshev polynomials of degree n + 1, which is given by T n+1 (x) = cos ((n + 1) arccos x) (39) so that T 0 (x) = 1; T 1 (x) = x; T 2 (x) = 2x 2 −1; T 3 (x) = 4x 3 −3x 2 , . . . . (40) 40 5.2 Chebyshev polynomials The Chebyshev polynomials of the first kind, T n , and the second kind, U n , are defined by the standard formulas T 0 (x) = 1, T 1 (x) = x; (41) T n (x) = 2xT n−1 (x) −T n−2 (x), n = 2, 3, ...; (42) U 0 (x) = 1, U 1 (x) = 2x; (43) U n (x) = 2xU n−1 (x) −U n−2 (x), n = 2, 3, ... . (44) If |x| ≤ 1, then the following representations hold: T n (x) = cos(narccos x), n = 0, 1, ...; (45) U n (x) = 1 √ 1 −x 2 sin ((n + 1) arccos x), n = 0, 1, . . . . (46) For x = ±1, a transition to the limit should be performed in formula (46). The Chebyshev polynomials are coupled by the relationships U n (x) = 1 n + 1 T n+1 (x), n = 0, 1, ... . (47) The Chebyshev polynomials are orthogonal functions with a weight on segment [−1, 1]: 1 _ −1 T n (x)T m (x) 1 √ 1 −x 2 dx = _ ¸ _ ¸ _ 0, n = m π/2, n = m = 0 π, n = m = 0 ; (48) 1 _ −1 U n (x)U m (x) √ 1 −x 2 dx = _ 0, n = m π/2, n = m . (49) Thus, the Chebyshev polynomials of the first kind are orthogonal on [−1, 1] with weight 1/ √ 1 −x 2 and the Chebyshev polynomials of the second kind are orthogonal on [−1, 1] with weight √ 1 −x 2 . Problem 5.1 Calculate the quantities ||P n −f|| using uniform and L 2 -norms in the interval [a, b] if (a) f(x) = x 2 −3x + 2; a = 0, b = 1, n = 1, P 1 (x) = −3x + 2; (b) f(x) = x 3 −2; a = −1, b = 1, n = 2, P 2 (x) = x 2 −2; (c) f(x) = √ x + 4; a = −4, b = 0, n = 1, P 1 (x) = 0.5x + 2. Problem 5.2 Construct reference-polynomial P 0 (x) ≡ P 0 = const of degree 0 at the reference set of 2 points, draw the graph and find reference deviation (with three decimal places) for the function f(x). (a) f(x) = x 3 + 2; x 0 = 0, x 1 = 1. (b) f(x) = sin πx; x 0 = 0.25, x 1 = 0.5. (c) f(x) = √ x; x 0 = 0, x 1 = 2. 41 Problem 5.4 Find coefficients λ i and reference deviation H at the reference set of n + 2 points, forming the uniform grid with the step h, for the function f(x). (a) f(x) = x 2 + 1; x 0 = 0, h = 0.1, n = 2. (b) f(x) = sin πx; x 0 = 0, h = 0.25, n = 1. (c) f(x) = 2 x ; x 0 = 0, h = 1, n = 1. Problem 5.5 Find analytically the polynomial of best approximation of degree 1 for Example 5.2 using the approach presented at the end of Example 5.1. 6 Numerical Methods in Linear Algebra. Gauss Elimi- nation A linear system of n equations in n unknowns x 1 , x 2 , . . ., x n is a set of n equations E 1 , E 2 , . . ., E n of the form E 1 : a 11 x 1 + a 12 x 2 + . . . + a 1n x n = b 1 E 2 : a 21 x 1 + a 22 x 2 + . . . + a 2n x n = b 2 . . . . . . . . E n : a n1 x 1 + a n2 x 2 + . . . + a nn x n = b n The system is called homogeneous if all b j = 0; otherwise it is inhomogeneous. We can rewrite (50) in the vector notation Ax = b, (50) where A = [a jk ] is the n ×n coefficient matrix of system (50) A = _ ¸ ¸ ¸ _ a 11 a 12 . . . a 1n a 21 a 22 . . . a 2n . . . . . . a n1 a n2 . . . a nn _ ¸ ¸ ¸ _ , and x = _ ¸ _ x 1 . . . x n _ ¸ _ , b = _ ¸ _ b 1 . . . b n _ ¸ _ are column vectors. The augmented matrix ˜ A of system (50) is the n ×(n + 1) matrix ˜ A = [Ab] = _ ¸ ¸ ¸ _ a 11 . . . a 1n b 1 a 21 . . . a 2n b 2 . . . . . . a n1 . . . a nn b n _ ¸ ¸ ¸ _ . A solution of system (50) is a set of n numbers x 1 , x 2 , . . ., x n that satisfy all n equations. They form the solution vector x. 42 The three elementary row operations can be applied to individual equations in a system without altering the solution of the system: (1) interchange the position of any two rows (i.e. equations); (2) multiply a row (i.e. equation) by a non-sero number; (3) replace a row by the sum of the row and a scalar multiple of another row . 6.1 Gauss elimination Example 6.1 Consider the application of Gauss elimination to solving the equation system E 1 : 3x 1 + 6x 2 + 9x 3 = 3 E 2 : 2x 1 + (4 + p)x 2 + 2x 3 = 4 E 3 : −3x 1 −4x 2 −11x 3 = −5 with the 3 ×3 coefficient matrix A = _ ¸ _ 3 6 9 2 4 + p 2 −3 −4 −11 _ ¸ _, and the column vectors x = _ ¸ _ x 1 x 2 x 3 _ ¸ _, b = _ ¸ _ 3 4 −5 _ ¸ _ . The augmented matrix ˜ A of system (51) is the 3 ×4 matrix ˜ A = [Ab] = _ ¸ _ 3 6 9 3 2 4 + p 2 4 −3 −4 −11 −5 _ ¸ _, and (51) can be written in the form _ ¸ _ 3 6 9 2 4 + p 2 −3 −4 −11 _ ¸ _ _ ¸ _ x 1 x 2 x 3 _ ¸ _ = _ ¸ _ 3 4 −5 _ ¸ _ . Gauss elimination to transform an augmented matrix to upper triangular form To do this, we will perform only the elementary row operations listed above, in three stages. At the first stage, we write the initial augmented matrix and choose the pivot element 3, denoted by 3 and situated in the first column of the first row. At the second stage, we make the elements of column 1 in rows 2 and 3 zero by (i) dividing row 1 by the pivot 3 and then (ii) subtracting a suitable mutiple of the modified row 1 (that is, the modified row 1 times 2) from row 2 and 43 (iii) adding a suitable mutiple of the modified row 1 (that is, the modified row 1 times 3) to row 3. Then we select the next pivot, p, situated in the second column of the new second row. To circumvent problems connected with a small p, we interchange the row in question (second) with the row with the elements of largest modulus in the column below the pivot (third row). This is called partial pivoting. Then at the third stage, we make the elements of column 2 in row 3 zero by (i) dividing row 2 by the pivot 2 and then (ii) subtracting a suitable mutiple of the modified row 2 (that is, the modified row 2 times p) from row 3. At the fourth stage, we thus obtain an upper triangular matrix. Unknowns x 1 , x 2 , and x 3 are determined by the substitution in the reversed order. Gauss elimination to transform an augmented matrix to upper triangular form Stage 1: Initial matrix _ ¸ _ row1 3 6 9 3 row2 2 4 + p 2 4 row3 −3 −4 −11 −5 _ ¸ _ . The element 3 in the first column of the first row is the pivot. The first equation is the pivot equation. Stage 2: Reduce col 1 of row 2 and 3 to zero _ ¸ _ oldrow1 3 6 9 3 oldrow2 - 2( oldrow1 )/3 0 p −4 2 oldrow3 +3( oldrow1 )/3 0 2 −2 −2 _ ¸ _, Interchange rows 2 and 3 _ ¸ _ oldrow1 3 6 9 3 oldrow3 0 2 −2 −2 oldrow2 0 p −4 2 _ ¸ _ . The element 2 in the second column of the second row is the pivot. The second equation is the pivot equation. Stage 3: Reduce col 2 of row 3 to zero _ ¸ _ oldrow1 3 6 9 3 oldrow2 0 2 −2 −2 oldrow3 - p( oldrow2 )/2 0 0 −4 + p 2 + p _ ¸ _ , Stage 4: Finding unknowns x 1 , x 2 , and x 3 by the substitution in the reversed order. 44 For p = 0, the system with the upper triangular matrix has the form 3x 1 + 6x 2 + 9x 3 = 3 2x 2 −2x 3 = −2 −4x 3 = 2 The substitution in the reversed order beginning from the third equation yields the solution x 1 = (3 −6 · (−1.5) −9 · (−0.5))/3 = 5.5 x 2 = 0.5(−2 + 2 · (−0.5)) = −1.5 x 3 = −0.5 Example 6.2 Gauss elimination to transform an augmented 2 ×3 matrix to upper triangular form Consider the application of Gauss elimination to solving the equation system E 1 : 2x 1 + 4x 2 = 1 E 2 : 6x 1 + 3x 2 = 2 with the 2 ×2 coefficient matrix A = _ 2 4 6 3 _ , and the column vectors x = _ x 1 x 2 _ , b = _ 1 2 _ . The system can be written in the form _ 2 4 6 3 _ _ x 1 x 2 _ = _ 1 2 _ . and the augmented matrix ˜ A of this system is the 2 ×3 matrix ˜ A = [Ab] = _ 2 4 1 6 3 2 _ . Perform Gauss elimination. Stage 1: Initial matrix We do pivoting by interchanging row 2 and 1 to obtain the augmented matrix with the maximum element a 11 ˜ A = [Ab] = _ row1 6 3 2 row2 2 4 1 _ , 45 or the system 6x 1 + 3x 2 = 2 2x 1 + 4x 2 = 1 The (maximum) element a 11 = 6 in the first column of the first row is the pivot. The first equation is the pivot equation. Stage 2: Reduce col 1 of row 2 to zero _ oldrow1 6 3 2 oldrow2 - 2( oldrow1 )/6 0 3 1/3 _ . Stage 3: Finding unknowns x 1 and x 2 by the substitution in the reversed order. The system with the upper triangular matrix obtained at Stage 2 has the form 6x 1 + 3x 2 = 2 3x 2 = 1/3 The substitution in the reversed order beginning from the second equation yields the solution x 1 = (2 −3 · (1/9))/6 = 5/18 x 2 = 2/18 Verify the result by the Cramer rule: det _ 2 4 6 3 _ = −18, x 1 = − 1 18 det _ 1 4 2 3 _ = 5/18, x 2 = − 1 18 det _ 2 1 6 2 _ = 2/18. 6.2 Application of MATLAB for solving linear equation systems The MATLAB operators \ and / perform matrix ’division’ and have identical effects. Thus to solve a linear equation system Ax = b we may write either A\ b or b /A . . In the latter case the solution x is expressed as a row rather than a column vector. The operators \ or / selects an appropriate algorithm dependent on the form of the matrix A. We note two of them: (i) if A is a triangular matrix the system is solved by the back or forward substitution alone; (ii) if A is a square matrix, general LU decomposition is applied which is equivalent to Gauss elimination in terms of elementary row operations. 46 Problem 6.1 Find the solution of the linear equation system 3x 1 + 5x 2 = 5.9 x 1 −4x 2 = 20.1 by Gauss elimination. Solution. The system has the form E 1 : 3x 1 + 5x 2 = 5.9 E 2 : x 1 −4x 2 = 20.1 with the 2 ×2 coefficient matrix A = _ 3 5 1 −4 _ , and the right-hand side column vector b = _ 5.9 20.1 _ . The system can be written in the form _ 3 5 1 −4 _ _ x 1 x 2 _ = _ 5.9 20.1 _ . and the augmented matrix ˜ A of this system is the 2 ×3 matrix ˜ A = [Ab] = _ 3 5 5.9 1 −4 20.1 _ . Perform Gauss elimination. Stage 1: Initial matrix We do pivoting by interchanging columns 2 and 1 to obtain the augmented matrix with the maximum element a 11 ˜ A = [Ab] = _ row1 5 3 5.9 row2 −4 1 20.1 _ , or the system 5x 2 + 3x 1 = 5.9 −4x 2 + x 1 = 20.1 The (maximum) element a 11 = 5 in the first column of the first row is the pivot. The first equation is the pivot equation. Stage 2: Reduce col 1 of row 2 to zero _ oldrow1 5 3 5.9 oldrow2 +4( oldrow1 )/5 0 3.4 24.82 _ . 47 Stage 3: Finding unknowns x 1 and x 2 by the substitution in the reversed order. The system with the upper triangular matrix obtained at Stage 2 has the form 5x 2 + 3x 1 = 5.9 3.4x 1 = 24.82 The substitution in the reversed order beginning from the second equation yields the solution x 2 = (5.9 −3 · (7.3))/5 = −3.2 x 1 = 7.3 Problem 6.2 Find the solution of the linear equation system 6x 2 + 13x 3 = 61 6x 1 −8x 3 = −38 13x 1 −8x 2 = 79 by Gauss elimination. Solution. Stage 1: Initial matrix _ ¸ _ row1 0 6 13 61 row2 6 0 −8 −38 row3 13 −8 0 79 _ ¸ _ . We do pivoting by interchanging rows 3 and 1 to obtain the augmented matrix with the maxi- mum element a 11 = 13 _ ¸ _ row1 13 −8 0 79 row2 6 0 −8 −38 row3 0 6 13 61 _ ¸ _ . The element 13 in the first column of the first row is the pivot. Now the first equation is the pivot equation. Stage 2: Reduce col 1 of row 2 and 3 to zero We do it only for row 2 because col 1 of row 3 is zero: _ ¸ _ oldrow1 13 −8 0 79 newrow2 = oldrow2 - 6( oldrow1 )/13 0 48/13 = 3.692308 −8 −74.461538 oldrow3 0 6 13 61 _ ¸ _ , Now we do pivoting by interchanging newrows 3 and 2 to obtain the augmented matrix with the maximum element a 22 = 6: 48 Interchange rows 2 and 3 _ ¸ _ oldrow1 13 −8 0 79 oldrow3 0 6 13 61 newrow2 0 3.692308 −8 −74.461538 _ ¸ _ . The element 6 in the second column of the second row is the pivot. The second equation is the pivot equation. Stage 3: Reduce col 2 of row 3 to zero _ ¸ _ oldrow1 13 −8 0 79 oldrow2 0 6 13 61 oldrow3 - (48/13) ( oldrow2 )/6 0 0 −16 −112 _ ¸ _, Stage 4: Finding unknowns x 1 , x 2 , and x 3 by the substitution in the reversed order. The system with the upper triangular matrix has the form 13x 1 −8x 2 = 79 6x 2 + 13x 3 = 61 −16x 3 = −112 The substitution in the reversed order beginning from the third equation yields the solution x 1 = (79 −(−8) · (−5))/13 = 3 x 2 = (61 −13 · 7)/6 = −5 x 3 = 7 7 Numerical Methods in Linear Algebra. Solution by Iteration 7.1 Convergence. Matrix norms Consider linear system (50) of n equations in n unknowns with matrix A. The matrix norm is given by one of the following expressions ||A|| = ¸ ¸ ¸ _ n j=1 n k=1 c 2 jk (Frobenius norm) ||A|| = max k n j=1 |c jk | (Column sum norm) ||A|| = max j n k=1 |c jk | (Row sum norm). The number cond (A) = ||A|| · ||A −1 || 49 is called the condition number. When it is very large, the solution of the linear equation system Ax = b will be very sensitive to small changes in b An iteration method method for solving Ax = b is said to converge for an initial x (0) if the corresponding iterative sequence x (0) , x (1) , . . ., x (n) , . . . converges to a solution of the given system with respect to one of the matrix norms. 7.2 Jacobi iteration Starting with the system Ax = b we rearrange it to the form x = b + (I −A)x where I = _ ¸ ¸ ¸ _ 1 0 . . . 0 0 1 . . . 0 . . . . . . 0 0 . . . 1 _ ¸ ¸ ¸ _ , is the identity matrix and define the Jacobi iteration as x (m+1) = b + (I −A)x (m) This method converges for every x (0) if and only if the spectral radius of I −A equal to its maximum eigenvalue is less than 1. The condition ||I −A|| < 1 is sufficient for the convergence of the Jacobi iteration. Example 7.1 Find the solution of the linear equation system 0.5x 1 −0.25x 2 = 1 −0.25x 1 + 0.5x 2 = 1 by the Jacobi iteration. Solution. The matrix of the system is A = _ 0.5 −0.25 −0.25 0.5 _ . I −A = _ 0.5 0.25 0.25 0.5 _ = aT, T = _ 2 1 1 2 _ , a = 1 4 . The Jacobi iteration is given by x (m+1) = b + aTx (m) . 50 7.3 Gauss–Seidel iteration Example 7.2 Consider a linear system of n = 4 equations x 1 −0.25x 2 −0.25x 3 = 50 −0.25x 1 + x 2 −0.25x 4 = 50 −0.25x 1 + x 3 −0.25x 4 = 25 −0.25x 2 −0.25x 3 + x 4 = 25 Rewrite this system in the form x 1 = 0.25x 2 + 0.25x 3 + 50 x 2 = 0.25x 1 + 0.25x 4 + 50 x 3 = 0.25x 1 + 0.25x 4 + 25 x 4 = 0.25x 2 + 0.25x 3 + 25 Choose an initial guess x = _ ¸ ¸ ¸ ¸ ¸ _ x (0) 1 x (0) 2 x (0) 3 x (0) 4 _ ¸ ¸ ¸ ¸ ¸ _ = _ ¸ ¸ ¸ _ 100 100 100 100 _ ¸ ¸ ¸ _ = 100 _ ¸ ¸ ¸ _ 1 1 1 1 _ ¸ ¸ ¸ _ and compute from (51) an approximation (first iteration) x (1) 1 = 0.25x (0) 2 + 0.25x (0) 3 + 50 = 100 x (1) 2 = 0.25x (1) 1 + 0.25x (0) 4 + 50 = 100 x (1) 3 = 0.25x (1) 1 + 0.25x (0) 4 + 25 = 75 x (1) 4 = 0.25x (1) 2 + 0.25x (1) 3 + 25 = 68.75. The next step (second iteration) is x (2) 1 = 0.25x (1) 2 + 0.25x (1) 3 + 50 = 93.75 x (2) 2 = 0.25x (2) 1 + 0.25x (1) 4 + 50 = 90.62 x (2) 3 = 0.25x (2) 1 + 0.25x (1) 4 + 25 = 65.62 x (2) 4 = 0.25x (2) 2 + 0.25x (2) 3 + 25 = 64.06. The exact solution is x = _ ¸ ¸ ¸ _ x 1 x 2 x 3 x 4 _ ¸ ¸ ¸ _ = _ ¸ ¸ ¸ _ 87.5 87.5 62.5 62.5 _ ¸ ¸ ¸ _ Describe the algorithm of the Gauss–Seidel iteration. Assume that all the matrix diagonal elements a jj = 1: A = I +L +U 51 where L and U are the lower and upper triangular matrices with the zero main diagonal. Substituting this representation into Ax = b yields Ax = (I +L +U)x = b or x = b −Lx −U)x which produces the Gauss–Seidel iteration method x (m+1) = b −Lx (m+1) −U)x (m) (a jj = 1) Rewrite the Gauss–Seidel iterations as (I +L)x (m+1) = b −Ux (m) . Multiplying by (I +L) −1 from the left, we obtain x (m+1) = Cx (m) + (I +L) −1 b. where the iteration matrix C = −(I +L) −1 U. The Gauss–Seidel iteration method converges for every x (0) if and only if all eigenvalues of C (spectral radius) are less than 1. The condition ||C|| < 1 is sufficient for the convergence of the Gauss–Seidel iteration method. 7.4 Least squares Consider approximating f(x) by a quadratic polynomial ˜ f(x) = a 1 + a 2 x + a 3 x 2 on the interval 0 ≤ x ≤ 1. To this end, choose a 1 , a 2 , and a 3 that minimize the root-mean-square (rms) error E = __ 1 0 [a 1 + a 2 x + a 3 x 2 −f(x)] 2 dx _ 1/2 . The resulting linear equation system is satisfied by the optimum choice of a 1 , a 2 , and a 3 . They are determined from ∂G ∂a i = 0, i = 1, 2, 3, G = E 2 . We have, denoting R(x) = a 1 + a 2 x + a 3 x 2 −f(x), ∂G ∂a 1 = ∂ ∂a 1 _ 1 0 R(x) 2 dx = _ 1 0 ∂ ∂a 1 R(x) 2 dx = _ 1 0 2R(x)R a 1 (x)dx = 2 _ 1 0 R(x)dx, ∂G ∂a 2 = _ 1 0 2R(x)R a 2 (x)dx = 2 _ 1 0 xR(x)dx, 52 ∂G ∂a 3 = _ 1 0 2R(x)R a 3 (x)dx = 2 _ 1 0 x 2 R(x)dx, Introduce the moments of f(x) I j = _ 1 0 x j f(x)dx, j = 0, 1, 2, and calculate _ 1 0 R(x)dx = a 1 + a 2 1 2 + a 3 1 3 −I 0 , _ 1 0 xR(x)dx = a 1 1 2 + a 2 1 3 + a 3 1 4 −I 1 , and _ 1 0 x 2 R(x)dx = a 1 1 3 + a 2 1 4 + a 3 1 5 −I 2 , Then, the optimum conditions ∂G ∂a j = 0, j = 0, 1, 2, yield the linear equation system with the Hilbert matrix a 1 + a 2 1 2 + a 3 1 3 = I 0 a 1 1 2 + a 2 1 3 + a 3 1 4 = I 1 a 1 1 3 + a 2 1 4 + a 3 1 5 = I 2 . or Ha = I with H = _ ¸ _ 1 1 2 1 3 1 2 1 3 1 4 1 3 1 4 1 5 _ ¸ _ . I = _ ¸ _ I 0 I 1 I 2 _ ¸ _, a = [a 1 , a 2 , a 3 ]. Problem 7.1 Find the solution of the linear equation system 5x 1 + x 2 + 2x 3 = 19 x 1 + 4x 2 −2x 3 = −2 2x 1 + 3x 2 + 8x 3 = 39 by the Gauss–Seidel iteration. Solution. 53 The matrix of the system is A = _ ¸ _ 5 1 2 1 4 −2 2 3 8 _ ¸ _ . Rewrite the system as x 1 = −1/5x 2 −2/5x 3 + 19/5 x 2 = −1/4x 1 + 1/2x 3 −1/2 x 3 = −1/4x 1 −3/8x 2 + 39/8 and compute the first iteration choosing an initial guess x = x (0) = _ ¸ ¸ _ x (0) 1 x (0) 2 x (0) 3 _ ¸ ¸ _ = _ ¸ _ 1 1 1 _ ¸ _ x (1) 1 = −1/5x (0) 2 −2/5x (0) 3 + 19/5 = −1/5 −2/5 + 19/5 = 16/5 x (1) 2 = −1/4x (1) 1 + 1/2x (0) 3 −1/2 = (−1/4)16/5 + 1/2 −1/2 = −4/5 x (1) 3 = −1/4x (1) 1 −3/8x (1) 2 + 39/8 = (−1/4)16/5 + (3/8)(4/5) + 39/8 = 35/8 The second iteration is x (2) 1 = −1/5x (1) 2 −2/5x (1) 3 + 19/5 x (2) 2 = −1/4x (2) 1 + 1/2x (2) 3 −1/2 x (2) 3 = −1/4x (2) 1 −3/8x (2) 2 + 39/8 Problem 7.2 Find a quadratic polynomial of best approximation for f(x) = e x using the least squares and the Hilbert matrix. 8 Numerical Methods for First-Order Differential Equa- tions We will consider numerical methods for solving the initial value problem y (x) = f(x, y), y(x 0 ) = y 0 . (51) When step-by-step methods are applied, the solution y(x) is determined at the mesh (or grid) points x 1 = x 0 + h, x 2 = x 0 + 2h, x 3 = x 0 + 3h, dots where h is the step size. 54 8.1 Euler method The Euler method is suggested by the use of the Taylor series: y(x + h) = y(x) + hy (x) + h 2 2 y (x) + dots, (52) y(x + h) ≈ y(x) + hy (x) = y(x) + hf(x, y). Then, at the first step, we compute y 1 = y 0 + hf(x 0 , y 0 ). At the second step, y 2 = y 1 + hf(x 1 , y 1 ), ans so on, so that the general formulas of the Euler method are y n+1 = y n + hf(x n , y n ), n = 0, 1, 2, . . . . (53) Example 5.1 Euler method Solve the initial value problem y (x) = x + y, y(0) = 0. with the step h = 0.2. Solution. In this case, the formulas of the Euler method are y n+1 = y n + 0.2(x n + y n ), n = 0, 1, 2, . . . . The exact solution is y(x) = e x −x −1. Denoting z n = 0.2(x n + y n ), y = Exact values, = Error we obtain the table of computed quantities n x n y n z n y 0 0.0 0.000 0.000 0.000 0.000 1 0.2 0.000 0.040 0.021 0.021 2 0.4 0.040 0.088 0.092 0.052 3 0.6 0.128 0.146 0.222 0.094 4 0.8 0.274 0.215 0.426 0.152 55 8.2 Improved Euler method This method is based on the substitution of y (x) = f(x, y(x)) (54) from (51) into (52): y(x + h) = y(x) + hf + h 2 2 f + h 3 6 f + . . . , where f = f x + f y y = f x + f y f, and the further derivatives are more cumbersome. The general formulas of the improved Euler method are as follows: at the first step, we compute y ∗ n+1 = y n + f(x n , y n ), n = 0, 1, 2, . . . ; (55) at the second step, we compute y n+1 = y n + h 2 [f(x n , y n ) + f(x n+1 , y ∗ n+1 )], n = 0, 1, 2, . . . . (56) or x n+1 = x n + h k 1 = hf(x n , y n ) k 2 = hf(x n+1 , y n + k 1 ) (57) y n+1 = y n + 1/2(k 1 + k 2 ) (58) Example 5.2 Improved Euler method Solve the initial value problem y (x) = x + y, y(0) = 0. with the step h = 0.2. Solution. In this case, the formulas of the improved Euler method are k 1 = 0.2(x n + y n ), k 1 = hf(x n , y n ); k 2 = 0.2(x n + 0.2 + y n + 0.2(x n + y n )), k 2 = hf(x n+1 , y ∗ n+1 , y ∗ n+1 = y n + k 1 ; Setting z n = 0.22(x n + y n ) + 0.02, y = Exact values, = Error we obtain the table of computed quantities n x n y n z n y 0 0.0 0.0000 0.0200 0.0000 0.0000 1 0.2 0.0200 0.0684 0.0214 0.014 2 0.4 0.0884 0.1274 0.0918 0.0034 3 0.6 0.2158 0.1995 0.2221 0.0063 4 0.8 0.4153 0.2874 0.4255 0.0102 56 8.3 Runge–Kutta methods The general formulas of the Runge–Kutta method of fourth order are as follows: k 1 = hf(x n , y n ) k 2 = hf(x n + 1/2h, y n + 1/2k 1 ) k 3 = hf(x n + 1/2h, y n + 1/2k 2 ) (59) k 4 = hf(x n + h, y n + k 3 ) x n+1 = x n + h y n+1 = y n + 1/6(k 1 + 2k 2 + 2k 3 + k 4 ) (60) Example 5.3 Classical Runge–Kutta method Solve the initial value problem y (x) = x + y, y(0) = 0. with the step h = 0.2. Solution. In this case, the formulas of the classical Runge–Kutta method are k 1 = 0.2(x n + y n ), k 2 = 0.2(x n + 0.1 + y n + 0.5k 1 ), k 3 = 0.2(x n + 0.1 + y n + 0.5k 2 ), k 4 = 0.2(x n + 0.2 + y n + k 3 ). By inserting the formulas for k 1 and k 2 into the respective expressions for k 2 , k 3 , and k 4 we have k 2 = 0.22(x n + y n ) + 0.22, k 3 = 0.222(x n + y n ) + 0.022, k 4 = 0.2444(x n + y n ) + 0.0444, and obtain y n+1 = y n + 1/6(k 1 + 2k 2 + 2k 3 + k 4 ) = y n + z n , where z n = 0.2214(x n + y n ) + 0.0214. Setting y = e x −x −1 = exact values, = 10 6 × error of y n we obtain the table of computed quantities n x n y n z n y 0 0.0 0.000000 0.021400 0.000000 0 1 0.2 0.021400 0.070418 0.021403 3 2 0.4 0.091818 0.130289 0.091825 7 3 0.6 0.222106 0.203414 0.222119 11 4 0.8 0.425521 0.292730 0.425541 20 57 Problem 8.1 Solve the initial value problem y + 0.1y = 0, y(0) = 2 with the step h = 0.1. Solution. In this case, we have y = −0.1y, The exact solution is y(x) = 2e −0.1x The formulas of the Euler method are y n+1 = y n −0.1 · 0.1y n = 0.99y n , n = 0, 1, 2, . . . , y 0 = 2. Denote x n = nh = 0.1n, y = y(x n ) Exact values, = Error = y(x n ) −y n . We have (performing calculations with 6D) y 1 = y 0 −0.01y 0 = 2 −0.02 = 1.98; = y(x 1 ) −y 1 = 2e −0.1·0.1 −1.98 = 1.980100 −1.980000 = 0.000100. y 2 = y 1 −0.01y 1 = 1.980 −0.0198 = 1.782; = y(x 2 ) −y 2 = 2e −0.1·0.2 −1.782 = 1.960397 −1.960200 = 0.000197. In general, y n+1 = 0.99y n = 0.99 2 y n−1 = . . . = 0.99 n+1 y 0 = 2 · 0.99 n+1 . We have (performing calculations with 6D) y 5 = 2 · 0.99 5 ; = y(x 5 ) −y 5 = 2e −0.1·0.5 −1.901980 = 1.902458 −1.901980 = 0.000478. y 10 = 2 · 0.99 10 ; = y(x 10 ) −y 10 = 2e −0.1 −1.901980 = 1.809674 −1.808764 = 0.000910. 58 Problem 8.2 Solve the initial value problem y = y, y(0) = 1 with the step h = 0.1. Solution. The exact solution is y(x) = e x . The formulas of the improved Euler method are as follows: x n+1 = x n + 0.1, n = 0, 1, 2, . . . , k 1 = hf(x n , y n ) = 0.1y n k 2 = hf(x n+1 , y n + k 1 ) = 0.1(y n + 0.1y n ) = 0.11y n , y n+1 = y n + 1/2(k 1 + k 2 ) = y n + 0.5(0.1y n + 0.11y n ) = 1.105y n We have y n+1 = 1.105y n = 1.105 2 y n−1 = . . . = 1.105 n+1 y 0 = 1.105 n+1 . so that, for example, y 5 = 1.105 5 = 1.647446; = y(x 5 ) −y 5 = e 0.5 −1.647446 = 1.648721 −1.647446 = 0.001274. y 10 = 1.105 10 = 2.714081; = y(x 10 ) −y 5 = e −2.714081 = 2.718281 −2.714081 = 0.004201. 8.4 Numerical solution of ordinary differential equations using MAT- LAB In this section we will use the notations x, x(t) for the sought-for function and t, t 0 , etc. for its argument (sometimes referred to as time) instead of the corresponding quantities y, y(x) and x, x 0 , etc. used in the previous section. MATLAB uses Runge–Kutta–Fehlberg methods to solve the initial-value problems for ODEs. The solution is computed in a finite number of points where the spacing depends on the solution. Fewer points are used in intervals where the solution is smooth and more points where the solution varies rapidly. [t,X] = ode23(str,t0,tt,x0) computes the solution of the ODE or the system of ODEs given by the string str. The solution is given in the vector t containing the t-values, and matrix X containing the solutions at these points. If the problem is scalar the result is returned to vector x. Solutions are computed from t0 to tt and x0 = x(t0) are the initial values. The system is determined using the function defined in the M-file str, which is required to have two parame- ters, scalar a and vector x, and to return vector x’ (the derivative). For a scalar ODE, x and x are scalars. 59 The approximation has a relative error of 10 −3 , at most. Example 8.4 Solve the initial value problem x = −x 2 , x(0) = 1. Solution. We create the function xprim1 stored in the M-file xprim1.m function z = xprim1(t,x) z = –x.∗x; Then we call the MATLAB solver [t,x] = ode23(’xprim1’,0,1,1); Finally we can plot the solution plot(t,x,’-’,t,x,’o’); xlabel(’time t0 = 0, tt = 1’); ylabel(’x values x(0) = 1’); Example 8.5 Solve the initial value problem x = x 2 , x(0) = 1. Solution. We create the function xprim2 stored in the M-file xprim2.m function z = xprim2(t,x) z = x.∗x; Then we call the MATLAB solver [t,x] = ode23(’xprim2’,0,0.95,1); and plot the solution plot(t,x,’-’,t,x,’o’); xlabel(’time t0 = 0, tt = 0.95’); ylabel(’x values x(0) = 1’); Note that the points where MATLAB computes the solution are denser in the area with high derivative. Example 8.6 Solve the initial value problem for the system of ODEs x 1 = x 1 −0.1x 1 x 2 + 0.01t x 2 = −x 2 + 0.02x 1 x 2 + 0.04t x 1 (0) = 30 x 2 (0) = 20 60 We create the function xprim3 stored in the M-file xprim3.m function xprim = xprim3(t,x) xprim(1) = x(1) – 0.1∗x(1)∗x(2)+ 0.01∗t; xprim(2) = –x(2) – 0.02∗x(1)∗x(2) + 0.04∗t; Then we call the MATLAB solver [t,x] = ode23(’xprim3’,0,20,[30;20]); and plot the solution plot(t,x); xlabel(’time t0 = 0, tt = 20’); ylabel(’x values x1(0) = 30, x2(0) = 20’); It is also possible to plot x 1 as a function of x 2 plot(x(:,2), x(:,1)); 61