The theory of the optimization algorithms reached maturity both in elaborating very sophisticated algorithms and in their complexity studies. For the unconstrained Newton optimization algorithm as well as for the barrier method's algorithms some closed formulas of the number of iterations necessary to be taken in order to get a specified accuracy solution are known. These formulas are obtained for problems satisfying very strong convexity conditions. Special results are obtained for self-concordant functions. Even if this theoretical result is very beautiful and interesting, it is of no practical value because testing the strong convexity conditions for a real optimization problem is more difficult than the original optimization one.
The purpose of this monograph is to build up a science of empiricism in analysing of the optimization algorithms as an alternative to the very sophisticated theoretical developments of this area. The central idea is that the analysis of the controlled numerical experiments in the context of the essence of the algorithm can give deep insights into the optimization algorithms complexity. Basically, the idea is to link the theoretical developments of the optimization algorithms with the empirical results given by the controlled numerical experiments.
The monograph is structured in 5 chapters. The first one has an introductory character. Some comments on the concept of "problem" are made. It is shown that any formalized problem can be reformulated as an optimization one. At the same time, we show that any optimization algorithm is a dynamic system. Chapter 2 has a special character, being dedicated to present the evolution of rationalism and empiricism along a period of 25 centuries. The criticism of theoretical development of optimization algorithms is given in chapter 3. This criticism refers to the Newton algorithm for unconstrained optimization and the barrier algorithm for constrained optimization. In chapter 4 the science of empiricism in analysis of optimization algorithms is defined as the science of controlled computational experiments. A criticism of testing processes of optimization algorithms represents our starting point for the definition of this science. The last chapter presents the basic elements of the science of controlled computational experiments in the context of the optimization algorithms. We emphasize some problems concerning the design of numerically stable optimization algorithms, identification of limits of algorithms, as well as the design of controlled numerical experiments. The monograph is a proposal for a new approach of analysing the optimization algorithms in which the theoretical developments are linked together with controlled numerical experiments in order to get an insight into the convergence and complexity of optimization algorithms.
 

                                                                                                                                                   Neculai Andrei