Pierre Borne is Professor at École Centrale de Lille in France. He has received honorary degrees from the University of Moscow, Russia, the Politehnica University of Bucharest, Romania and the University of Waterloo, Canada. He is a Fellow of the IEEE.Dumitru Popescu is Professor at Politehnica University of Bucharest, Romania in the fields of Advanced Control and Systems Optimization and also Associate Professor at major universities of France and Italy. He is Director of the Research Center in Automatics, Process Control and Computers (APCC) affiliated to Politehnica University of Bucharest, a member of the IFAC Technical Committees for the Control of Bioprocesses and Chemical Processes and a corresponding member of the Romanian Academy of Technical Sciences.Florin Gh. Filip is a researcher in the fields of applied IT and decision support systems. He was elected as a member of the Romanian Academy in 1991 and president of the “IT” section in 2011. He was Vice President of the Romanian Academy from 2001 to 2010.Dan Stefanoiu is Professor in the fields of Signal Processing and System Identification at Politehnica University of Bucharest. In 2002 he was elected as a member of the American Romanian Academy of Sciences and Arts (ARA).

• Foreword ixPreface xiList of Acronyms xiiiChapter 1. Linear Programming 11.1. Objective of linear programming 11.2. Stating the problem 11.3. Lagrange method 41.4. Simplex algorithm 51.4.1. Principle 51.4.2. Simplicial form formulation 51.4.3. Transition from one simplicial form to another 71.4.4. Summary of the simplex algorithm 91.5. Implementation example 111.6. Linear programming applied to the optimization of resource allocation 131.6.1. Areas of application 131.6.2. Resource allocation for advertising 131.6.3. Optimization of a cut of paper rolls 161.6.4. Structure of linear program of an optimal control problem 17Chapter 2. Nonlinear Programming 232.1. Problem formulation 232.2. Karush–Kuhn–Tucker conditions 242.3. General search algorithm 262.3.1. Main steps 262.3.2. Computing the search direction 292.3.3. Computation of advancement step 332.4. Monovariable methods 332.4.1. Coggin’s method (of polynomial interpolation) 342.4.2. Golden section method 362.5. Multivariable methods 392.5.1. Direct search methods 392.5.2. Gradient methods 57Chapter 3. Dynamic Programming 1013.1. Principle of dynamic programming 1013.1.1. Stating the problem 1013.1.2. Decision problem 1013.2. Recurrence equation of optimality 1023.3. Particular cases 1043.3.1. Infinite horizon stationary problems 1043.3.2. Variable horizon problem 1043.3.3. Random horizon problem 1043.3.4. Taking into account sum-like constraints 1053.3.5. Random evolution law 1063.3.6. Initialization when the final state is imposed 1063.3.7. The case when the necessary information is not always available 1073.4. Examples 1073.4.1. Route optimization 1073.4.2. The smuggler problem 109Chapter 4. Hopfield Networks 1154.1. Structure 1154.2. Continuous dynamic Hopfield networks 1174.2.1. General problem 1174.2.2. Application to the traveling salesman problem 1214.3. Optimization by Hopfield networks, based on simulated annealing 1234.3.1. Deterministic method 1234.3.2. Stochastic method 125Chapter 5. Optimization in System Identification 1315.1. The optimal identification principle 1315.2. Formulation of optimal identification problems 1325.2.1. General problem 1325.2.2. Formulation based on optimization theory 1335.2.3. Formulation based on estimation theory (statistics) 1365.3. Usual identification models 1385.3.1. General model 1385.3.2. Rational input/output (RIO) models 1405.3.3. Class of autoregressive models (ARMAX) 1425.3.4. Class of state space representation models 1455.4. Basic least squares method 1465.4.1. LSM type solution 1465.4.2. Geometric interpretation of the LSM solution 1515.4.3. Consistency of the LSM type solution 1545.4.4. Example of application of the LSM for an ARX model 1575.5. Modified least squares methods 1585.5.1. Recovering lost consistency 1585.5.2. Extended LSM 1625.5.3. Instrumental variables method 1645.6. Minimum prediction error method 1685.6.1. Basic principle and algorithm 1685.6.2. Implementation of the MPEM for ARMAX models 1715.6.3. Convergence and consistency of MPEM type estimations 1745.7. Adaptive optimal identification methods 1755.7.1. Accuracy/adaptability paradigm 1755.7.2. Basic adaptive version of the LSM 1775.7.3. Basic adaptive version of the IVM 1825.7.4. Adaptive window versions of the LSM and IVM 183Chapter 6. Optimization of Dynamic Systems 1916.1. Variational methods 1916.1.1. Variation of a functional 1916.1.2. Constraint-free minimization 1926.1.3. Hamilton canonical equations 1946.1.4. Second-order conditions 1956.1.5. Minimization with constraints 1956.2. Application to the optimal command of a continuous process, maximum principle 1966.2.1. Formulation 1966.2.2. Examples of implementation 1986.3. Maximum principle, discrete case 2066.4. Principle of optimal command based on quadratic criteria 2076.5. Design of the LQ command 2106.5.1. Finite horizon LQ command 2106.5.2. The infinite horizon QL command 2176.5.3. Robustness of the LQ command 2216.6. Optimal filtering 2246.6.1. Kalman–Bucy predictor 2256.6.2. Kalman–Bucy filter 2316.6.3. Stability of Kalman–Bucy estimators 2346.6.4. Robustness of Kalman–Bucy estimators 2356.7. Design of the LQG command 2396.8. Optimization problems connected to quadratic linear criteria 2456.8.1. Optimal control by state feedback 2456.8.2. Quadratic stabilization 2486.8.3. Optimal command based on output feedback 249Chapter 7. Optimization of Large-Scale Systems 2517.1. Characteristics of complex optimization problems 2517.2. Decomposition techniques 2527.2.1. Problems with block-diagonal structure 2537.2.2. Problems with separable criteria and constraints 2677.3. Penalization techniques 2837.3.1. External penalization technique 2847.3.2. Internal penalization technique 2857.3.3. Extended penalization technique 286Chapter 8. Optimization and Information Systems 2898.1. Introduction 2898.2. Factors influencing the construction of IT systems 2908.3. Approaches 2928.4. Selection of computing tools 2968.5. Difficulties in implementation and use 2978.6. Evaluation 2978.7. Conclusions 298Bibliography 299Index 307

Pierre Borne is Professor at École Centrale de Lille in France. He has received honorary degrees from the University of Moscow, Russia, the Politehnica University of Bucharest, Romania and the University of Waterloo, Canada. He is a Fellow of the IEEE.Dumitru Popescu is Professor at Politehnica University of Bucharest, Romania in the fields of Advanced Control and Systems Optimization and also Associate Professor at major universities of France and Italy. He is Director of the Research Center in Automatics, Process Control and Computers (APCC) affiliated to Politehnica University of Bucharest, a member of the IFAC Technical Committees for the Control of Bioprocesses and Chemical Processes and a corresponding member of the Romanian Academy of Technical Sciences.Florin Gh. Filip is a researcher in the fields of applied IT and decision support systems. He was elected as a member of the Romanian Academy in 1991 and president of the “IT” section in 2011. He was Vice President of the Romanian Academy from 2001 to 2010.Dan Stefanoiu is Professor in the fields of Signal Processing and System Identification at Politehnica University of Bucharest. In 2002 he was elected as a member of the American Romanian Academy of Sciences and Arts (ARA).

• Foreword ixPreface xiList of Acronyms xiiiChapter 1. Linear Programming 11.1. Objective of linear programming 11.2. Stating the problem 11.3. Lagrange method 41.4. Simplex algorithm 51.4.1. Principle 51.4.2. Simplicial form formulation 51.4.3. Transition from one simplicial form to another 71.4.4. Summary of the simplex algorithm 91.5. Implementation example 111.6. Linear programming applied to the optimization of resource allocation 131.6.1. Areas of application 131.6.2. Resource allocation for advertising 131.6.3. Optimization of a cut of paper rolls 161.6.4. Structure of linear program of an optimal control problem 17Chapter 2. Nonlinear Programming 232.1. Problem formulation 232.2. Karush–Kuhn–Tucker conditions 242.3. General search algorithm 262.3.1. Main steps 262.3.2. Computing the search direction 292.3.3. Computation of advancement step 332.4. Monovariable methods 332.4.1. Coggin’s method (of polynomial interpolation) 342.4.2. Golden section method 362.5. Multivariable methods 392.5.1. Direct search methods 392.5.2. Gradient methods 57Chapter 3. Dynamic Programming 1013.1. Principle of dynamic programming 1013.1.1. Stating the problem 1013.1.2. Decision problem 1013.2. Recurrence equation of optimality 1023.3. Particular cases 1043.3.1. Infinite horizon stationary problems 1043.3.2. Variable horizon problem 1043.3.3. Random horizon problem 1043.3.4. Taking into account sum-like constraints 1053.3.5. Random evolution law 1063.3.6. Initialization when the final state is imposed 1063.3.7. The case when the necessary information is not always available 1073.4. Examples 1073.4.1. Route optimization 1073.4.2. The smuggler problem 109Chapter 4. Hopfield Networks 1154.1. Structure 1154.2. Continuous dynamic Hopfield networks 1174.2.1. General problem 1174.2.2. Application to the traveling salesman problem 1214.3. Optimization by Hopfield networks, based on simulated annealing 1234.3.1. Deterministic method 1234.3.2. Stochastic method 125Chapter 5. Optimization in System Identification 1315.1. The optimal identification principle 1315.2. Formulation of optimal identification problems 1325.2.1. General problem 1325.2.2. Formulation based on optimization theory 1335.2.3. Formulation based on estimation theory (statistics) 1365.3. Usual identification models 1385.3.1. General model 1385.3.2. Rational input/output (RIO) models 1405.3.3. Class of autoregressive models (ARMAX) 1425.3.4. Class of state space representation models 1455.4. Basic least squares method 1465.4.1. LSM type solution 1465.4.2. Geometric interpretation of the LSM solution 1515.4.3. Consistency of the LSM type solution 1545.4.4. Example of application of the LSM for an ARX model 1575.5. Modified least squares methods 1585.5.1. Recovering lost consistency 1585.5.2. Extended LSM 1625.5.3. Instrumental variables method 1645.6. Minimum prediction error method 1685.6.1. Basic principle and algorithm 1685.6.2. Implementation of the MPEM for ARMAX models 1715.6.3. Convergence and consistency of MPEM type estimations 1745.7. Adaptive optimal identification methods 1755.7.1. Accuracy/adaptability paradigm 1755.7.2. Basic adaptive version of the LSM 1775.7.3. Basic adaptive version of the IVM 1825.7.4. Adaptive window versions of the LSM and IVM 183Chapter 6. Optimization of Dynamic Systems 1916.1. Variational methods 1916.1.1. Variation of a functional 1916.1.2. Constraint-free minimization 1926.1.3. Hamilton canonical equations 1946.1.4. Second-order conditions 1956.1.5. Minimization with constraints 1956.2. Application to the optimal command of a continuous process, maximum principle 1966.2.1. Formulation 1966.2.2. Examples of implementation 1986.3. Maximum principle, discrete case 2066.4. Principle of optimal command based on quadratic criteria 2076.5. Design of the LQ command 2106.5.1. Finite horizon LQ command 2106.5.2. The infinite horizon QL command 2176.5.3. Robustness of the LQ command 2216.6. Optimal filtering 2246.6.1. Kalman–Bucy predictor 2256.6.2. Kalman–Bucy filter 2316.6.3. Stability of Kalman–Bucy estimators 2346.6.4. Robustness of Kalman–Bucy estimators 2356.7. Design of the LQG command 2396.8. Optimization problems connected to quadratic linear criteria 2456.8.1. Optimal control by state feedback 2456.8.2. Quadratic stabilization 2486.8.3. Optimal command based on output feedback 249Chapter 7. Optimization of Large-Scale Systems 2517.1. Characteristics of complex optimization problems 2517.2. Decomposition techniques 2527.2.1. Problems with block-diagonal structure 2537.2.2. Problems with separable criteria and constraints 2677.3. Penalization techniques 2837.3.1. External penalization technique 2847.3.2. Internal penalization technique 2857.3.3. Extended penalization technique 286Chapter 8. Optimization and Information Systems 2898.1. Introduction 2898.2. Factors influencing the construction of IT systems 2908.3. Approaches 2928.4. Selection of computing tools 2968.5. Difficulties in implementation and use 2978.6. Evaluation 2978.7. Conclusions 298Bibliography 299Index 307

Pierre Borne is Professor at École Centrale de Lille in France. He has received honorary degrees from the University of Moscow, Russia, the Politehnica University of Bucharest, Romania and the University of Waterloo, Canada. He is a Fellow of the IEEE.Dumitru Popescu is Professor at Politehnica University of Bucharest, Romania in the fields of Advanced Control and Systems Optimization and also Associate Professor at major universities of France and Italy. He is Director of the Research Center in Automatics, Process Control and Computers (APCC) affiliated to Politehnica University of Bucharest, a member of the IFAC Technical Committees for the Control of Bioprocesses and Chemical Processes and a corresponding member of the Romanian Academy of Technical Sciences.Florin Gh. Filip is a researcher in the fields of applied IT and decision support systems. He was elected as a member of the Romanian Academy in 1991 and president of the “IT” section in 2011. He was Vice President of the Romanian Academy from 2001 to 2010.Dan Stefanoiu is Professor in the fields of Signal Processing and System Identification at Politehnica University of Bucharest. In 2002 he was elected as a member of the American Romanian Academy of Sciences and Arts (ARA).

• Foreword ixPreface xiList of Acronyms xiiiChapter 1. Linear Programming 11.1. Objective of linear programming 11.2. Stating the problem 11.3. Lagrange method 41.4. Simplex algorithm 51.4.1. Principle 51.4.2. Simplicial form formulation 51.4.3. Transition from one simplicial form to another 71.4.4. Summary of the simplex algorithm 91.5. Implementation example 111.6. Linear programming applied to the optimization of resource allocation 131.6.1. Areas of application 131.6.2. Resource allocation for advertising 131.6.3. Optimization of a cut of paper rolls 161.6.4. Structure of linear program of an optimal control problem 17Chapter 2. Nonlinear Programming 232.1. Problem formulation 232.2. Karush–Kuhn–Tucker conditions 242.3. General search algorithm 262.3.1. Main steps 262.3.2. Computing the search direction 292.3.3. Computation of advancement step 332.4. Monovariable methods 332.4.1. Coggin’s method (of polynomial interpolation) 342.4.2. Golden section method 362.5. Multivariable methods 392.5.1. Direct search methods 392.5.2. Gradient methods 57Chapter 3. Dynamic Programming 1013.1. Principle of dynamic programming 1013.1.1. Stating the problem 1013.1.2. Decision problem 1013.2. Recurrence equation of optimality 1023.3. Particular cases 1043.3.1. Infinite horizon stationary problems 1043.3.2. Variable horizon problem 1043.3.3. Random horizon problem 1043.3.4. Taking into account sum-like constraints 1053.3.5. Random evolution law 1063.3.6. Initialization when the final state is imposed 1063.3.7. The case when the necessary information is not always available 1073.4. Examples 1073.4.1. Route optimization 1073.4.2. The smuggler problem 109Chapter 4. Hopfield Networks 1154.1. Structure 1154.2. Continuous dynamic Hopfield networks 1174.2.1. General problem 1174.2.2. Application to the traveling salesman problem 1214.3. Optimization by Hopfield networks, based on simulated annealing 1234.3.1. Deterministic method 1234.3.2. Stochastic method 125Chapter 5. Optimization in System Identification 1315.1. The optimal identification principle 1315.2. Formulation of optimal identification problems 1325.2.1. General problem 1325.2.2. Formulation based on optimization theory 1335.2.3. Formulation based on estimation theory (statistics) 1365.3. Usual identification models 1385.3.1. General model 1385.3.2. Rational input/output (RIO) models 1405.3.3. Class of autoregressive models (ARMAX) 1425.3.4. Class of state space representation models 1455.4. Basic least squares method 1465.4.1. LSM type solution 1465.4.2. Geometric interpretation of the LSM solution 1515.4.3. Consistency of the LSM type solution 1545.4.4. Example of application of the LSM for an ARX model 1575.5. Modified least squares methods 1585.5.1. Recovering lost consistency 1585.5.2. Extended LSM 1625.5.3. Instrumental variables method 1645.6. Minimum prediction error method 1685.6.1. Basic principle and algorithm 1685.6.2. Implementation of the MPEM for ARMAX models 1715.6.3. Convergence and consistency of MPEM type estimations 1745.7. Adaptive optimal identification methods 1755.7.1. Accuracy/adaptability paradigm 1755.7.2. Basic adaptive version of the LSM 1775.7.3. Basic adaptive version of the IVM 1825.7.4. Adaptive window versions of the LSM and IVM 183Chapter 6. Optimization of Dynamic Systems 1916.1. Variational methods 1916.1.1. Variation of a functional 1916.1.2. Constraint-free minimization 1926.1.3. Hamilton canonical equations 1946.1.4. Second-order conditions 1956.1.5. Minimization with constraints 1956.2. Application to the optimal command of a continuous process, maximum principle 1966.2.1. Formulation 1966.2.2. Examples of implementation 1986.3. Maximum principle, discrete case 2066.4. Principle of optimal command based on quadratic criteria 2076.5. Design of the LQ command 2106.5.1. Finite horizon LQ command 2106.5.2. The infinite horizon QL command 2176.5.3. Robustness of the LQ command 2216.6. Optimal filtering 2246.6.1. Kalman–Bucy predictor 2256.6.2. Kalman–Bucy filter 2316.6.3. Stability of Kalman–Bucy estimators 2346.6.4. Robustness of Kalman–Bucy estimators 2356.7. Design of the LQG command 2396.8. Optimization problems connected to quadratic linear criteria 2456.8.1. Optimal control by state feedback 2456.8.2. Quadratic stabilization 2486.8.3. Optimal command based on output feedback 249Chapter 7. Optimization of Large-Scale Systems 2517.1. Characteristics of complex optimization problems 2517.2. Decomposition techniques 2527.2.1. Problems with block-diagonal structure 2537.2.2. Problems with separable criteria and constraints 2677.3. Penalization techniques 2837.3.1. External penalization technique 2847.3.2. Internal penalization technique 2857.3.3. Extended penalization technique 286Chapter 8. Optimization and Information Systems 2898.1. Introduction 2898.2. Factors influencing the construction of IT systems 2908.3. Approaches 2928.4. Selection of computing tools 2968.5. Difficulties in implementation and use 2978.6. Evaluation 2978.7. Conclusions 298Bibliography 299Index 307