Intrinsically time-dependent or unsteady systems, where steady-state analysis is not applicable, constitute an important class of engineering problems. Such systems often arise in fluid dynamics for problems that are inherently dynamic, such as flapping flight, or problems where a steady-state solution does not exist, such as separated flow. Design or control of such systems calls for the solution of time-dependent PDE-constrained optimization problems. In this work, a framework for shape optimization and optimal control for unsteady, viscous CFD problems is developed. Since the optimization functionals require reliable accuracy in the computed output quantities and their gradients, the governing equations are discretized using a high-order discontinuous Galerkin (DG) spatial discretization and high-order temporal discretization. In the case of unsteady shape optimization or optimal control, the geometry of interest will in general be evolving with time, resulting in a deforming fluid domain. This is handled by a high-order accurate mapping-based Arbitrary Lagrangian-Eulerian (ALE) approach. High-order temporal integration is achieved through a Diagonally-Implicit Runge Kutta scheme ensuring the global discretization is high-order. The fully discrete adjoint method is used to compute gradients of CFD output functionals with respect to the optimization parameters. The proposed high-order unsteady CFD optimization framework will be demonstrated on viscous CFD test problems. Future work will focus on reducing the relatively high computational cost associated with unsteady optimization for CFD problems by using a reduced-order model as a surrogate for the high-order DG equations. The benefits of using a high-order discretization will be preserved by considering a globally convergent sequence of such reduced optimization problems.