Optimization Techniques in Engineering: Foundations, Algorithms and Applications - From Classical Calculus to Metaheuristics and Machine-Learning-Assisted Design

About the Book

What the book tries to do. It tries to build, from first principles, an understanding of why optimization algorithms work - and, just as importantly, when they fail. Every algorithm in this book is presented in four layers:

1. The idea. A plain-language statement of the trick that makes the method work, usually accompanied by a picture. If you remember nothing else from a chapter, remember the picture.
2. The mathematics. The derivation, the conditions under which it is valid, and the convergence behaviour you should expect.
3. The procedure. A step-by-step algorithm you could implement, in pseudocode or in Python, without further help.
4. The engineering. A worked example drawn from a real discipline - a truss, a heat exchanger, a distribution network, a controller - carried through to numbers.

Who the book is for. The book is self-contained enough for a first course at the senior undergraduate level. It assumes calculus, linear algebra, and the patience to follow an algebraic argument; it does not assume prior exposure to operations research or to numerical analysis. Chapters 1-3 build the necessary background. For a postgraduate course, Chapters 7, 8, 12, 13, and 17-20 provide considerably more depth, and the later chapters on robust, stochastic and surrogate-assisted optimization take the reader to the edge of current practice.
How the book is organised. The material is arranged in six parts.

• Part I - Foundations (Chapters 1-3) establishes vocabulary, the mathematical machinery of convexity and optimality, and - most neglected of all topics - the art of formulating an engineering problem so that it is solvable.
• Part II - Classical Deterministic Methods (Chapters 4-8) covers single-variable search, gradient-based multivariable methods, direct search, and the classical treatment of constraints through Lagrange multipliers, the Karush-Kuhn-Tucker conditions, penalty methods, and sequential quadratic programming.
• Part III - Linear, Network and Discrete Optimization (Chapters 9-13) develops linear programming and the simplex method, duality and sensitivity, network models, integer programming, and dynamic programming.
• Part IV - Metaheuristic and Nature-Inspired Methods (Chapters 14-17) treats genetic algorithms, swarm intelligence, simulated annealing, tabu search, differential evolution, and multi-objective optimization.
• Part V - Advanced Topics (Chapters 18-20) covers geometric and quadratic programming, optimization under uncertainty, and surrogate-based design.
• Part VI - Applications and Practice (Chapters 21-24) puts everything to work on structural, mechanical, electrical, chemical and civil case studies, surveys the software landscape, and closes with the fast-moving frontier where machine learning meets optimization.

A note on rigour. I have not hidden the proofs, but I have not let them run the book either. Where a proof illuminates the algorithm - the orthogonality of successive steepest-descent directions, the geometric content of the KKT conditions, the reason the simplex method terminates - it is given in full. Where a proof is long and the result is intuitive, it is stated carefully and referenced. The engineer's task is to know what a theorem guarantees, not necessarily to re-derive it at three in the morning.