Optimization (Methods)

Intro

• Optimization in Sciences and Engineering: Running Examples (fluid mechanics, bio-, energy systems)
• Examples of formulations, flavors: discrete-continuous, convex-non-convex, multi-objective, stochastic, chance-constrained, multi-level (e.g. games), function –> functionals (functions of functions)= calculus of variations
• What does solving optimization problem mean? Complexity. Exact vs Approximate. Analytic,  automatic, numerical.

Convex Optimization (continuous)

• Examples – Linear, Quadratic, Geometric, Semidefinite
• Duality, change of variables, transformations and other tricks
• Methods of solutions (main ideas), how to use it in non-convex world  (link to next section)
• Link to discrete optimization studies (in the “statistics” portion)

Non-convex Optimization

• find local solution, count local solutions, feasibility
• convexification/approximations (regularizations) – examples.
• change of variables … other tricks, examples

More complex optimization problems

• Robust and Stochastic Optimizations -  formulations, methods of solutions
• Optimization of functionals = Calculus of variations.  Example: optimization over paths.

Bonus

• Lyapunov function and stability: conditions of stability, examples of Lyapunov functions , more on dynamical systems, Hamiltonian systems, ODEs, linear systems, linearization
• Optimal Control.  Bellman-Hamilton-Jacobi. Dynamic Programming. Linear-Quadratic-Gaussian control problems.