Infinite Horizon Optimal Control: Deterministic and Stochastic Systems

This monograph deals with various classes of deterministic and stochastic continuous time optimal control problems that are defined over unbounded time intervals. For these problems the performance criterion is described by an improper integral and it is possible that, when evaluated at a given admissible element, this criterion is unbounded. To cope with this divergence new optimality concepts, referred to here as overtaking optimality, weakly overtaking optimality, agreeable plans, etc. , have been proposed. The motivation for studying these problems arises primarily from the economic and biological sciences where models of this type arise naturally. Indeed, any bound placed on the time hori­ zon is artificial when one considers the evolution of the state of an economy or species. The responsibility for the introduction of this interesting class of problems rests with the economists who first studied them in the modeling of capital accumulation processes. Perhaps the earliest of these was F. Ramsey [152] who, in his seminal work on the theory of saving in 1928, considered a dynamic optimization model defined on an infinite time horizon. Briefly, this problem can be described as a Lagrange problem with unbounded time interval. The advent of modern control theory, particularly the formulation of the famous Maximum Principle of Pontryagin, has had a considerable impact on the treat­ ment of these models as well as optimization theory in general.

Table of Contents:
1 Dynamical Systems with Unbounded Time Interval in Engineering, Ecology and Economics.- 1.1 Introduction.- 1.2 The regulator problem.- 1.3 The pest control problem and other problems of optimal control of interacting species.- 1.4 The optimal economic growth problem.- 1.5 Definition of optimality on an unbounded time interval.- 1.6 Uniformly optimal solutions are agreeable.- 2 Necessary Conditions and Sufficient Conditions for Optimality.- 2.1 Introduction.- 2.2 The maximum principle with a finite horizon.- 2.3 The optimality principle.- 2.4 A maximum principle for an infinite horizon control problem.- 2.5 Sufficient conditions for overtaking optimality.- 3 Asymptotic Stability and the Turnpike Property in Some Simple Control Problems.- 3.1 Introduction.- 3.2 Saddle point property of the Hamiltonian in a convex problem of Lagrange. Implications on local asymptotic stability of optimally controlled systems.- 3.3 An exact turnpike property: Optimal fish harvest.- 3.4 Use of a phase diagram for a one-state-variable control problem: The simple optimal economic growth model.- 4 Global Asymptotic Stability and Existence of Optimal Trajectories for Infinite Horizon Autonomous Convex Systems.- 4.1 Introduction.- 4.2 The class of systems considered.- 4.3 Convergence toward the Von Neumann Set for weakly overtaking trajectories.- 4.4 The turnpike property.- 4.5 Global asymptotic stability for extremal trajectories.- 4.6 A Lyapunov function approach for GAS of optimal trajectories.- 4.7 Sufficient conditions for overtaking optimality.- 4.8 Existence of optimal trajectories.- 4.9 Overtaking optimality under relaxed assumptions.- 5 The Reduction to Finite Rewards.- 5.1 Introduction.- 5.2 The Property R.- 5.3 The connection between continuous and discrete time control systems.- 5.4Existence of a reduction to finite rewards.- 5.5 A representation formula and turnpike properties of optimal controls.- 5.6 Systems with unbounded rewards and with discounting factors.- 5.7 Infinite horizon tracking of periodic signals.- 5.8 Optimal trajectories and turnpike properties of infinite horizon autonomous nonconvex systems.- 5.9 Two special cases: Scalar systems, and integrands in a separated form.- 6 Asymptotic Stability with a Discounted Criterion; Global and Local Analysis.- 6.1 Introduction.- 6.2 Modified Hamiltonian systems.- 6.3 Cass-Shell conditions for GAS of modified Hamiltonian systems.- 6.4 Brock-Sheinkman conditions for GAS of modified Hamiltonian systems.- 6.5 Another useful condition for GAS.- 6.6 Neighboring extremals, the second variation and analysis of local asymptotic stability of a stationary point, using the optimal linear quadratic regulator problem.- 6.7 The turnpike property for finite horizon optimal control problems with discounting.- 7 Turnpike Properties and Existence of Overtaking Optimal Solutions for Classes of Nonautonomous Nonconvex Control Problems.- 7.1 Introduction.- 7.2 G-supported trajectories.- 7.3 Carathéodory’s method for finite horizon optimal control problems.- 7.4 Carathéodory’s method for infinite horizon optimal control problems.- 7.5 The growth condition (?) and the compactness of the set of admissible trajectories.- 7.6 Upper closure and the existence of strongly optimal solutions.- 7.7 The existence of overtaking optimal solutions.- 8 Control of Systems with Integrodifferential Equations.- 8.1 Introduction.- 8.2 The basic model.- 8.3 Linear hereditary operators and an upper closure theorem.- 8.4 Existence of overtaking optimal solutions.- 8.5 Examples.- 9 Extensions to Distributed Parameter Systems.- 9.1Introduction.- 9.2 Examples.- 9.3 Semigroups of operators and linear control systems.- 9.4 The optimal control problem.- 9.5 The turnpike properties.- 9.6 Existence of overtaking optimal solutions.- 9.7 More on the examples.- 9.8 The extension to systems with distributed parameters and boundary controls.- 10 Stochastic Control with the Overtaking Criterion.- 10.1 Introduction.- 10.2 The reduction to finite costs and the infinite-horizon Bellman equation.- 10.3 Infinite-horizon stochastic tracking.- 10.4 Optimal control of nonlinear diffusions in ?n.- 10.5 On almost-sure overtaking optimality.- 11 Maximum Principle and Turnpike Properties for Systems with Random Modal Jumps.- 11.1 Introduction.- 11.2 Optimal control under random stopping time.- 11.3 Turnpike properties.- 11.4 Piecewise Deterministic Control Systems.- 11.5 Global turnpike property for constant jump rates 3.