Optimization (Methods)

Intro

  • Optimization in Sciences and Engineering: Running Examples (fluid mechanics, bio-, energy systems)
  • Examples of formulations, flavors: discrete-continuous, convex-non-convex, multi-objective, stochastic, chance-constrained, multi-level (e.g. games), function –> functionals (functions of functions)= calculus of variations
  • What does solving optimization problem mean? Complexity. Exact vs Approximate. Analytic,  automatic, numerical.

Convex Optimization (continuous)

  • Examples – Linear, Quadratic, Geometric, Semidefinite
  • Duality, change of variables, transformations and other tricks
  • Methods of solutions (main ideas), how to use it in non-convex world  (link to next section)
  • Link to discrete optimization studies (in the “statistics” portion)

Non-convex Optimization

  • find local solution, count local solutions, feasibility
  • convexification/approximations (regularizations) – examples.
  • change of variables … other tricks, examples

More complex optimization problems

  • Robust and Stochastic Optimizations -  formulations, methods of solutions
  • Optimization of functionals = Calculus of variations.  Example: optimization over paths.

Bonus

  • Lyapunov function and stability: conditions of stability, examples of Lyapunov functions , more on dynamical systems, Hamiltonian systems, ODEs, linear systems, linearization
  • Optimal Control.  Bellman-Hamilton-Jacobi. Dynamic Programming. Linear-Quadratic-Gaussian control problems.