Suppose that is a (continuously differentiable) function and is its local minimum. Pick an arbitrary vector . Since we are in the unconstrained case, moving away from in the direction of or cannot immediately take us outside . In other words, we have for all close enough to 0.

For a fixed , we can consider as a function of the real parameter , whose domain is some interval containing 0. Let us call this new function :

Since is a minimum of , it is clear that 0 is a minimum of . Passing from to is useful because is a function of a scalar variable and so its minima can be studied using ordinary calculus. In particular, we can write down the first-order Taylor expansion for around :

where represents ``higher-order terms" which go to 0 faster than as approaches 0, i.e.,

We claim that

To show this, suppose that . Then, in view of (1.6), there exists an small enough so that for , the absolute value of the fraction in (1.6) is less than . We can write this as

For these values of , (1.5) gives

If we further restrict to have the opposite sign to , then the right-hand side of (1.8) becomes 0 and we obtain . But this contradicts the fact that has a minimum at 0. We have thus shown that (1.7) is indeed true.

We now need to re-express this result in terms of the original function . A simple application of the chain rule from vector calculus yields the formula

where

is the

and this equals 0 by (1.7). Since was arbitrary, we conclude that

This is the

A point
satisfying this condition is called a *stationary point*.
The condition is ``first-order" because it is derived using the first-order
expansion (1.5).
We emphasize that the result
is valid when
and
is an interior point
of
.