1.2 Some background on finite-dimensional optimization

Consider a function Let be some subset of , which could be the entire . We denote by the standard Euclidean norm on .

A point
is a *local minimum* of
over
if there exists an
such that for all
satisfying
we
have

In other words, is a local minimum if in some ball around it, does not attain a value smaller than . Note that this refers only to points in ; the behavior of outside is irrelevant, and in fact we could have taken the domain of to be rather than .

If the inequality in (1.3)
is strict for
, then we have a *strict* local
minimum.
If (1.3) holds for *all*
, then the minimum is
*global* over
. By default, when we say ``a minimum" we mean a local minimum.
Obviously, a minimum need not be unique unless it is both strict and
global.

The notions of a (local, strict, global) *maximum* are defined similarly.
If a point is either a maximum or a minimum, it is called an *extremum*.
Observe that maxima of
are minima of
, so there is no need to develop separate results for
both. We focus on the minima, i.e., we view
as a *cost*
function to be minimized (rather than a profit to be maximized).

- 1.2.1 Unconstrained optimization
- 1.2.1.1 First-order necessary condition for optimality
- 1.2.1.2 Second-order conditions for optimality
- 1.2.1.3 Feasible directions, global minima, and convex problems

- 1.2.2 Constrained optimization