4.3.1.2 Time-dependent system and cost

The same idea of appending the state variable $x_{n+1}=t$ and passing to the system (4.39) can be applied when the original system's right-hand side $f$ and/or the running cost $L$ depend on $t$ . The Hamiltonian is now time-dependent:

$$H(t,x,u,p,p_0):=\langle p,f(t,x,u)\rangle +p_0L(t,x,u).$$ (4.40)

The previous discussion remains valid up to and including the equation $\dot p^*_{n+1}=\left.-{H}_{t}\right\vert _{*}$ but the latter no longer equals 0. Thus, $p^*_{n+1}$ and $\left.H\right\vert _{*}=-p^*_{n+1}$ are not constant any more. Instead, we have the differential equation

$${\frac{d}{dt}}\left.H\right\vert _{*}=\left.{H}_{t}\right\vert _{*}$$ (4.41)

with the boundary condition $\left.H\right\vert _{*}(t_f)=-p_{n+1}^*(t_f)$ . If the terminal time $t_f$ in the original problem is free, then the final value of $x_{n+1}$ is free and the transversality condition yields $p_{n+1}^*(t_f)=0$ . In this case we obtain $\left.H\right\vert _{*}(t_f)=0$ and, integrating (4.41), $\left.H\right\vert _{*}(t)=-\int_t^{t_f}\left.{H}_{t}\right\vert _{*}(s)ds.$ Note that (4.41) is consistent with the equation obtained in (3.43) in the context of the variational approach (although the middle portion of (3.43) does not apply here).

The same conclusion can be reached by following the proof of the maximum principle and verifying that it carries over to the time-varying scenario without major changes, except that the claim of Exercise 4.6 becomes invalid and only (4.41) can be established. We can appreciate, however, that in the present case the method of changing the variables is much simpler and more reliable.