4.2.4 Variational equation

We are now interested in how the difference between the trajectory $y$ arising from a spatial (needle) perturbation and the optimal trajectory $y^*$ propagates after the perturbation stops acting, i.e., for $t\ge b$ . To study this question, let us begin by writing

$$y(t)=y^*(t)+\varepsilon \psi(t)+o(\varepsilon )=:y(t,\varepsilon )$$ (4.16)

for $b\le t\le t^*$ , where $\psi:[b,t^*]\to\mathbb{R}^{n+1}$ is a quantity that we want to characterize and $\varepsilon$ is the same as in Section 4.2.3. We know from (4.14) that $\psi(b)$ exists and is given by

$$\psi(b)=\nu_b(w)a.$$ (4.17)

We now derive a differential equation that $\psi$ must satisfy on the time interval $(b,t^*]$ , where both $y$ and $y^*$ have the same corresponding control $u^*$ . It is clear from (4.16) that

$$\psi(t)={y}_{\varepsilon }(t,0)$$ (4.18)

(the existence of this partial derivative, and therefore of $\psi$ , for $t>b$ will become evident in a moment). Let us rewrite the system (4.7) as an integral equation:

$$y(t,\varepsilon )=y(b,\varepsilon )+\int_b^tg(y(s,\varepsilon ),u^*(s))ds.$$

Differentiating both sides of this equation with respect to $\varepsilon$ at $\varepsilon =0$ and using (4.16) with $t=b$ and (4.17), we obtain

$${y}_{\varepsilon }(t,0)=\nu_b(w)a+\int_b^t {g}_{y}(y(s,0),u^*(s)){y}_{\varepsilon }(s,0)ds$$

which, in view of (4.16) and (4.18), amounts to

$$\psi(t)=\nu_b(w)a+\int_b^t {g}_{y}(y^*(s),u^*(s))\psi(s)ds.$$

Taking the derivative with respect to $t$ , we conclude that $\psi$ satisfies the differential equation

$$\dot \psi={g}_{y}(y^*,u^*)\psi=\left.{g}_{y}\right\vert _{*}\psi.$$ (4.19)

We will use this equation to describe how spatial perturbations propagate with time. Pictorially, the role of $\psi$ is illustrated in Figure 4.6, with the understanding that the labels involving $\psi$ are accurate only up to terms of order $o(\varepsilon )$ .

Figure: Propagation of a spatial perturbation

The equation (4.19) can be written in the form

$$\dot \psi=A_*(t)\psi$$ (4.20)

where $A_*(t):=\left.{g}_{y}\right\vert _{*}(t)$ . This system is simply the linearization of the original system (4.7) around the optimal trajectory $y^*$ . Recall that a more detailed description of the system (4.7) is given by (4.5). Letting $(\eta^0,\eta)$ be the corresponding components of $\psi$ and writing out the variational equation (4.19) in terms of these components, we easily arrive at

$$\dot\eta^0 =\big.{\left({L}_{x}\right)^T}\big\vert _*\eta$$

$$\dot\eta =\left.{f}_{x}\right\vert _{*}\eta$$

(because $L$ and $f$ do not depend on $x^0$ ). Here $({L}_{x})^T$ is a row vector (the transpose of the gradient of $L$ with respect to $x$ ) and ${{f}_{x}}$ is an $n\times n$ matrix (the Jacobian matrix of $f$ with respect to $x$ ). The resulting more explicit formula for $A_*$ is

$$A_*(t)= \begin{pmatrix}0 & \big.{\left({L}_{x}\right)^T}\big\vert _* \\ 0 & \left.{f}_{x}\right\vert _{*} \end{pmatrix}.$$ (4.21)

The equation (4.19) is in fact the variational equation corresponding to the system (4.7), according to the terminology introduced in Section 2.6.2. The equation (3.27) in Section 3.4 is also similar and was similarly derived, although it served a different purpose (it described the effect of a small control perturbation on a given trajectory, while here we are studying the difference between two nearby trajectories with the same control). Also, we see that $\varepsilon$ in the definition of a needle perturbation essentially corresponds to $\alpha$ in the variational approach; we chose to use different symbols for these two parameters because of the different specific ways in which they are introduced.

The value of $\psi$ at the terminal time $t^*$ gives us an approximation of the terminal point $y(t^*)$ of the perturbed trajectory. Namely, from (4.16) evaluated at $t=t^*$ we have

$$y(t^*)=y^*(t^*)+\varepsilon \psi(t^*)+o(\varepsilon ).$$ (4.22)

Let us denote by $\Phi_*(\cdot,\cdot)$ the transition matrix for the linear time-varying system (4.20), so that

$$\psi(t^*)=\Phi_*(t^*,b)\psi(b).$$

The initial value $\psi(b)$ is given by (4.17), hence

$$\psi(t^*)=\Phi_*(t^*,b)\nu_b(w)a$$

where $\nu_b(w)$ was defined in (4.15). Plugging this expression into (4.22), we obtain

$$y(t^*)=y^*(t^*)+\varepsilon \Phi_*(t^*,b)\nu_b(w)a +o(\varepsilon ).$$ (4.23)

Let us introduce the notation

$$\delta(w,I):=\Phi_*(t^*,b)\nu_b(w)a$$ (4.24)

to arrive at the more compact formula

$$y(t^*)=y^*(t^*)+\varepsilon \delta(w,I)+o(\varepsilon )$$

(here the interval $I$ used for constructing the needle perturbation encodes information about the values of $a$ and $b$ ).