So, we can see the root cause of this stutter is moving some objects in Update and others in FixedUpdate. The simple fix is indeed to move all transforms in either Update or FixedUpdate.

However, this is where things get tricky. The common answer found among Unity developers is to put most game and movement logic in Update and use FixedUpdate only for the occasional physics task. While this has merits, including simplicity of use, it is problematic for many reasons. Perhaps most important is that your gameplay is frame rate dependent. This opens the door for plenty of gameplay affecting bugs and inconsistent behavior. Furthermore, it prevents determinism, which almost the entire real time strategy genre, for example, depends on. It also introduces problems when you need accelerated motion for an object, such as a character controller’s gravity. For those objects FixedUpdate should be used, but since other objects are moved in Update you’ll get stutter (see the standard assets first person character controller to observe this exact issue).

Therefore, a common and sometimes necessary alternative is to put all state and gameplay logic in a fixed timestep like FixedUpdate and strictly handle visuals and input logic in Update. This is not without its own challenges however. First off, you may want your physics steps to occur at a different rate from game logic ticks depending on your game. That can be resolved by implementing your own fixed timestep loop independent of FixedUpdate that ticks at whatever rate you wish. This is fairly simple to do and can give you a lot of control, allowing for fine tuned optimization. Next, inputs can be missed when read only in FixedUpdate, since multiple frames could occur between FixedUpdates, and only the last frame’s input survives to the following FixedUpdate. This particularly affects button up and down events, as they are only active for a single frame. The solution to this issue is buffering inputs by storing them each frame until they are all processed during the next FixedUpdate. Integrating this behaviour into whatever input controller you use is a fairly seamless way to do this and keeps the buffering in one place.

One larger issue, however, is that FixedUpdate is typically called less than the client frame rate, so moving objects don’t update their position as frequently as the screen is rendered. This makes the game somewhat choppy, though consistent. There are various ways to resolve this centering around using interpolation and extrapolation to fill the frames between FixedUpdates, smoothing the movement out. Interpolation, moving an object smoothly from one game state to the following state, is convenient in that it can be applied to most objects and work easily, but does introduce a fixedDeltaTime worth of latency. This latency is generally accepted however, and plenty of games, even twitch shooters and such, allow for this delay to gain smoothness. Extrapolation, predicting where an object will be next fixed step, avoids latency, but is inherently more difficult to get working seamlessly and comes with a performance cost.