Bayesian Approach to Global Optimization: Theory and Applications

*Et moi, ...si j'avait su comment en revcnir. One service mathematics has rendered the je o'y semis point alle.' human race. It has put common sense back Jules Verne where it beloogs. on the topmost shelf next to the dusty canister labelled 'discarded non- The series is divergent; therefore we may be sense', able to do something with it. Eric T. BclI O. Heaviside Mathematics is a tool for thought. A highly necessary tool in a world where both feedback and non- linearities abound. Similarly, all kinds of parts of mathematics serve as tools for other parts and for other sciences. Applying a simple rewriting rule to the quote on the right above one finds such statements as: 'One service topology has rendered mathematical physics ...'; 'One service logic has rendered com- puter science ...'; 'One service category theory has rendered mathematics ...'. All arguably true. And all statements obtainable this way form part of the raison d'etre of this series.

Table of Contents:
1 Global optimization and the Bayesian approach.- 1.1 What is global optimization?.- 1.2 Advantages of the Bayesian approach to global optimization.- 2 The conditions of Bayesian optimality.- 2.1 Introduction.- 2.2 Reduction to dynamic programming equations.- 2.3 The existence of a measurable solution.- 2.4 The calculation of conditional expectations.- 2.5 The one-step approximation.- 2.6 The adaptive Bayesian approach.- 3 The axiomatic non-probabilistic justification of Bayesian optimality conditions.- 3.1 Introduction.- 3.2 The linearity of the loss function.- 3.3 The existence of the unique a priori probability corresponding to subjective preferences.- 3.4 Optimal method under uncertainty.- 3.5 Nonlinear loss functions.- 4 Stochastic models.- 4.1 Introduction.- 4.2 Sufficient convergence conditions.- 4.3 The Gaussian field.- 4.4 Homogeneous Wiener field.- 4.5 A case of noisy observations.- 4.6 Estimation of parameters from dependent observations.- 5 Bayesian methods for global optimization in the Gaussian case.- 5.1 The one-step approximation.- 5.2 Adaptive models.- 5.3 Extrapolation models.- 5.4 Maximum likelihood models.- 5.5 The comparison of algorithms.- 5.6 The Bayesian approach to global optimization with linear constraints.- 5.7 The Bayesian approach to global optimization with nonlinear constraints.- 5.8 The Bayesian approach to multi-objective optimization.- 5.9 Interactive procedures and the Bayesian approach to global optimization.- 5.10 The reduction of multi-dimensional data.- 5.11 The stopping rules.- 6 The analysis of structure and the simplification of the optimization problems.- 6.1 Introduction.- 6.2 Structural characteristics and the optimization problem.- 6.3 The estimation of structural characteristics.- 6.4 The estimation of a simplification error.- 6.5 Examples of the estimates.- 7 The Bayesian approach to local optimization.- 7.1 Introduction.- 7.2 The one-dimensional Bayesian model.- 7.3 Convergence of the local Bayesian algorithm.- 7.4 Generalization of a multi-dimensional case.- 7.5 Convergence in the multi-dimensional case.- 7.6 The local Bayesian algorithm.- 7.7 Results of computer simulation.- 8 The application of Bayesian methods.- 8.1 Introduction.- 8.2 The optimization of an electricity meter.- 8.3 The optimization of vibromotors.- 8.4 The optimization of a shock-absorber.- 8.5 The optimization of a magnetic beam deflection system.- 8.6 The optimization of small aperture coupling between a rectangular waveguide and a microstrip line.- 8.7 The maximization of LSI yield by optimization of parameters of differential amplifier functional blocks.- 8.8 Optimization of technology to avoid waste in the wet-etching of printed circuit boards in iron-copper-chloride solutions.- 8.9 The optimization of pigment compounds.- 8.10 The least square estimation of electrochemical adsorption using observations of the magnitude of electrode impedance.- 8.11 Estimation of parameters of the immunological model.- 8.12 The optimization of nonstationary queuing systems.- 8.13 The analysis of structure of the Steiner problem.- 8.14 The estimation of decision making by intuition on the example of the Steiner problem.- 9 Portable FORTRAN software for global optimization.- 9.1 Introduction.- 9.2 Parameters.- 9.3 Methods available.- 9.4 Common blocks.- 9.5 The function.- 9.6 The main program.- 9.7 The example of the main program.- 9.8 Description of routines.- 9.9 BAYES1, the global Bayesian method by Mockus.- 9.10 UNT, the global method of extrapolation type by Zilinskas.- 9.11 LPMIN, the global method of uniform search by Sobolj, Shaltenis and Dzemyda.- 9.12 GLOPT, the global method of clustering type by Törn.- 9.13 MIG1, the global method of Monte Carlo (uniform random search).- 9.14 MIG2, the modified version of MIG 1.- 9.15 EXTR, the global one-dimensional method by Zilinskas.- 9.16 MIVAR4, the local method of variable metrics by Tieshis.- 9.17 REQP, the local method of recursive quadratic programming by Biggs.- 9.18 FLEXI, the local simplex method by Nelder and Mead.- 9.19 LBAYES, the local Bayesian method by Mockus.- 9.20 ANAL1, the method of analysis by structure by Shaltenis.- 9.21 Portability routines.- References.- Appendix 1 The software for global optimization for IMB/PC/XT/AT and compatibles.- Appendix 2 How the global optimization software can improve the performance of your CAD system.- Appendix 3 Machine dependent constants of portable FORTRAN.