Gradient methods
Frank-Wolfe and Projected gradient methods
Subgradient methods
Mirror descent
Block coordinate descent
Proximal gradient methods
Nesterov's accelerated methods
Operator splitting
Alternating direction methods of multiplier (ADMM)
Primal-dual proximal methods
Quasi-Newton methods / BFGS
Stochastic optimization
Variance reduction in stochastic optimization
Distributed optimization
Saddle-escaping algorithms