Nonlinear Systems and Optimization for the Chemical Engineer: Solving Numerical Problems

This third in a suite of four practical guides is an engineer's companion to using numerical methods for the solution of complex mathematical problems. It explains the theory behind current numerical methods and shows in a step-by-step fashion how to use them. The volume focuses on optimization from experimental to large-scale processes, detailing the algorithms needed to solve real-life problems. It describes the methods, innovative techniques and strategies that are all implemented in a well-established, freeware mathematical toolbox called BzzMath, which is developed and maintained by the authors. Numerous examples, sample codes, programs and applications are taken from a wide range of scientific and engineering fields, such as chemical engineering, electrical engineering, physics, medicine, and environmental science. As a result, engineers and scientists learn how to use the latest and most powerful numerical methods in their daily work. With additional online material including the latest version of BzzMath Library, installation tutorial, all examples and sample codes used in the book and a host of further examples.

Table of Contents:
Preface XI 1 Function Root-Finding 1 1.1 Introduction 1 1.2 Substitution Algorithms 4 1.3 Bolzano's Algorithm 7 1.4 Function Approximation 8 1.4.1 Newton's Method 9 1.4.2 The Secant Method 10 1.4.3 Regula Falsi Method 11 1.4.4 Muller's Method or Parabolic Interpolation 12 1.4.5 Hyperbolic Interpolation Method 13 1.4.6 Inverse Polynomial Interpolation Method 13 1.4.7 Inverse Rational Interpolation Method 14 1.5 Use of a Multiprocessor Machine with a Known Interval of Uncertainty 16 1.6 Search for an Interval of Uncertainty 17 1.7 Stop Criteria 17 1.8 Classes for Function Root-Finding 21 1.9 Case Studies 26 1.9.1 Calculation of the Volume of a Nonideal Gas 26 1.9.2 Calculation of the Bubble Point of Vapor-Liquid Equilibrium 28 1.9.3 Zero-Crossing Problem 30 1.9.4 Stationary Condition in a Gravity-Flow Tank 33 1.10 Tests for BzzFunctionRoot and BzzFunctionRootMP Classes 35 1.11 Some Caveats 39 2 One-Dimensional Optimization 41 2.1 Introduction 41 2.2 Measuring the Efficiency of the Search for the Minimum 45 2.3 Comparison Methods 46 2.4 Parabolic Interpolation 58 2.5 Cubic Interpolation 59 2.6 Gradient-Based Methods 59 2.7 Combination of Algorithms in a General Program 60 2.8 Parallel Computations 60 2.9 Search for the Interval of Uncertainty 61 2.10 Stop Criteria 61 2.11 Classes for One-Dimensional Minimization 62 2.12 Case Studies 69 2.12.1 Optimization of Unimodal Functions 69 2.12.2 Optimization of a Batch Reactor 71 2.12.3 Maximum Level in a Gravity-Flow Tank in Transient Conditions 74 2.13 Tests 77 3 Unconstrained Optimization 79 3.1 Introduction 79 3.1.1 Necessary and Sufficient Conditions 80 3.1.2 Quadratic Functions 81 3.1.3 Directions of Function Decrease 83 3.1.4 Comparison with the One-Dimensional Case 84 3.1.5 Classification of Methods 85 3.2 Heuristic Methods 86 3.2.1 Modified Hooke-Jeeves Method 88 3.2.2 The Rosenbrock Method 89 3.2.3 The Nelder-Mead Simplex Method 92 3.2.4 Robust Optnov Method Combined with the Simplex Method 95 3.3 Gradient-Based Methods 98 3.4 Conjugate Direction Methods 100 3.5 Newton's Method 105 3.6 Modified Newton Methods 109 3.6.1 Singular or Nonpositive Definite Hessian Matrix 110 3.6.2 Convergence Problems 116 3.6.3 One-Dimensional Search 118 3.6.4 Trust Region Methods 121 3.6.5 Use of Alternative Methods 124 3.7 Quasi-Newton Methods 126 3.8 Narrow Valley Effect 131 3.9 Stop Criteria 133 3.10 BzzMath Classes for Unconstrained Multidimensional Minimization 135 3.11 Case Study 141 3.11.1 Optimization of a Batch Reactor 142 3.11.2 Optimal Adiabatic Bed Reactors for Sulfur Dioxide with Cold Shot Cooling 144 3.11.3 Global Optimization 148 3.12 Tests 150 4 Large-Scale Unconstrained Optimization 153 4.1 Introduction 153 4.2 Collecting a Sparse Symmetric Matrix 154 4.3 Ordering the Hessian Rows and Columns 156 4.4 Quadratic Functions 161 4.5 Hessian Evaluation 171 4.6 Newton's Method 173 4.7 Inexact Newton Methods 173 4.8 Practical Preconditioners 180 4.9 openMP Parallelization 180 4.10 Class for Large-Scale Unconstrained Minimization 180 5 Robust Unconstrained Minimization 185 5.1 Introduction 185 5.2 One-Dimensional Minimization 186 5.3 Classes for One-Dimensional Robust Minimization 187 5.4 Examples in One-Dimensional Space 190 5.5 Examples in Multidimensional Space 199 5.6 Two-Dimensional Minimization 202 5.7 Classes for Robust Two-Dimensional Minimization 205 5.8 Examples for BzzMinimizationTwoVeryRobust Class 206 5.9 Multidimensional Robust Minimization 216 5.9.1 Outer Optimizer 217 5.9.2 Inner Optimizer 218 5.10 Class for Robust Multidimensional Minimization 219 6 Robust Function Root-Finding 225 6.1 Introduction 225 6.2 Class and Examples 225 7 Nonlinear Systems 235 7.1 Introduction 235 7.2 Comparing Nonlinear Systems to Other Iterative Problems 236 7.2.1 Comparison with Function Root-Finding 236 7.2.2 Comparison with the Multidimensional Optimization 238 7.3 Convergence Test 240 7.4 Substitution Methods 244 7.5 Minimization Methods 244 7.6 Jacobian Evaluation 245 7.7 Newton's Method 245 7.8 Gauss-Newton Method 248 7.9 Modified Newton Methods 250 7.9.1 Singular or Ill-Conditioned Jacobian 250 7.9.2 Convergence Problem 253 7.10 Newton's Method and Parallel Computations 256 7.11 Quasi-Newton Methods 257 7.12 Quasi-Newton Methods and Parallel Computing 260 7.13 Stop Criteria 261 7.13.1 Bounds, Constraints, and Discontinuities 261 7.14 Classes for Nonlinear System Solution with Dense Matrices 262 7.15 Tests for the BzzNonLinearSystem Class 268 7.16 Sparse and Large-Scale Systems 272 7.17 Large Linear System Solution with Iterative Methods 278 7.18 Classes for Nonlinear System Solution with Sparse Matrices 279 7.19 Continuation Methods 281 7.20 Solution of Certain Equations with Respect to Certain Variables 284 7.21 Case Studies 287 7.21.1 Heat Exchange in a Thermal Furnace 287 7.21.2 Calculation of Chemical Equilibria 289 7.21.3 Multiple Solutions in a CSTR 294 7.21.4 Critical Size of a Nuclear Reactor 296 7.21.5 Stationary Conditions for a Nonisothermal Continuous Stirred Tank Reactor 298 7.21.6 Vapor-Liquid Equilibrium: The Flash Separator 300 7.21.7 Boundary Value Problems 302 7.22 Special Cases 302 7.22.1 Vapor-Liquid Equilibrium: Distillation Column 302 7.22.2 Kinetic Postprocessor 303 7.23 Some Caveats 306 8 Underdimensioned Nonlinear Systems 313 8.1 Introduction 313 8.2 Underdimensioned Linear Systems 314 8.2.1 Null Space 314 8.2.2 Determination of One Solution 316 8.2.3 Projection Methods 327 8.2.4 Stable Gauss Factorization 331 8.3 Class for Underdimensioned Nonlinear System Solution 339 9 Constrained Minimization 343 9.1 Introduction 343 9.2 Equality Constraints 344 9.3 Equality and Inequality Constraints 346 9.4 Lagrangian Dual Problem 350 10 Linear Programming 355 10.1 Introduction 355 10.2 Basic Attic Method Concepts 357 10.3 Attic Method 358 10.3.1 Certain Important Peculiarities of the Attic Method 359 10.3.2 What Happens in a Generic Iteration 360 10.3.3 Selecting the New Feasible Point and the New Inequality Constraint 361 10.3.4 Selection of the Search Direction 361 10.4 Differences between the Attic Method and Traditional Approaches 363 10.4.1 A Simple Application of the Attic Method 367 10.4.2 Certain Advantages to the Attic Method 368 10.5 Explosion in the Number of Iterations 370 10.5.1 Selecting Constraints Rather Than Vertices 371 10.5.2 The Tile Effect 377 10.5.3 Wall Constraints and Roof Constraints 378 10.5.4 When a Constraint with l>0 Should also be Removed 380 10.6 Degeneracy 382 10.7 Duality 385 10.8 General Considerations 387 11 Quadratic Programming 389 11.1 Introduction 389 11.2 KKT Conditions for a QP Problem 392 11.3 Equality-Constrained QP 393 11.3.1 Solving the Full KKT System 394 11.3.2 Shur-Complement Method 397 11.3.3 Null Space Methods 399 11.4 Equality- and Inequality-Constrained Problems 404 11.5 Class for QP 406 11.6 Projection or Reduced Direction Search Methods for Bound-Constrained Problems 407 11.7 Equality, Inequality, and Bound Constraints 412 11.8 Tests 416 12 Constrained Minimization: Penalty and Barrier Functions 419 12.1 Introduction 419 12.2 Penalty Function Methods 419 12.2.1 Quadratic Penalty Function 421 12.2.2 Nonsmooth Exact Penalty Function 423 12.2.3 The Maratos Effect 426 12.2.4 Augmented Lagrangian Penalty Function 430 12.2.5 Bound-Constrained Formulation for Lagrangian Penalty Function 434 12.3 Barrier Function Methods 434 12.4 Mixed Penalty-Barrier Function Methods 437 13 Constrained Minimization: Active Set Methods 439 13.1 Introduction 439 13.2 Class for Constrained Minimization 441 13.3 Successive Linear Programming 455 13.4 Projection Methods 457 13.5 Reduced Direction Search Methods 461 13.6 Projection or Reduced Direction Search Methods for Bound-Constrained Problems 463 13.7 Successive Quadratic Programming or Projected Lagrangian Method 464 13.7.1 Selection of the Merit Function 467 13.7.2 Updating the Jacobian of the System 470 13.8 Narrow Valley Effect 471 13.9 The Nonlinear Constraints Effect 472 13.10 Tests 473 14 Parametric Continuation in Optimization and Process Control 477 14.1 Introduction 477 14.2 Algebraic Constraints 478 14.2.1 Distillation Column 478 References 481 Appendix A: Copyrights 487 Index 489

About the Author :
Guido Buzzi-Ferraris is full professor of process systems engineering at Politecnico die Milano, Italy, where he holds two courses: "Methods and Numerical Applications in Chemical Engineering" and "Regression Models and Statistics". He works on numerical analysis, statistics, differential systems, and optimization. He has authored books of international relevance on numerical analysis, such as "Scientific C++" edited by Addison-Wesley, and over than 200 papers on international magazines. He is the inventor and the developer of BzzMath library, which is currently adopted by academies, R&D groups, and industries. He is permanent member of the "EFCE Working Party - Computer Aided Process Engineering" since 1969 and editorial advisory board of "Computers & Chemical Engineering" since 1987. Flavio Manenti is assistant professor of process systems engineering at Politecnico di Milano, Italy. He obtained his academic degree and PhD at Politecnico di Milano, where he currently collaborates with Professor Buzzi-Ferraris. He holds courses on "Process Dynamics and Control of Industrial Processes" and "Supply Chain Optimization" and he works on numerical analysis, process control and optimization. He has also received international scientific awards, such as Memorial Burianec (Prague, CZ) and Excellence in Simulation (Lake Forest, CA, USA), for his research activities and scientific publications.