On a daily basis, humans interact with a vast range of objects and tools. A class of tasks, which can pose a serious challenge to our motor skills, are those that involve manipulating objects with internal degrees of freedom, such as when folding laundry or using a lasso. Here, we use the framework of optimal feedback control to make predictions of how humans should interact with such objects. We confirm the predictions experimentally in a two-dimensional object manipulation task, in which subjects learned to control six different objects with complex dynamics. We show that the non-intuitive behavior observed when controlling objects with internal degrees of freedom can be accounted for by a simple cost function representing a trade-off between effort and accuracy. In addition to using a simple linear, point-mass optimal control model, we also used an optimal control model, which considers the non-linear dynamics of the human arm. We find that the more realistic optimal control model captures aspects of the data that cannot be accounted for by the linear model or other previous theories of motor control. The results suggest that our everyday interactions with objects can be understood by optimality principles and advocate the use of more realistic optimal control models for the study of human motor neuroscience.

Humans are highly skilled at tool use. Simple tools have no internal degrees of freedom. For example, knowing the position and orientation of a hammer allows us to, in theory, predict the forces it will generate on our hand when we wield it and the consequences our actions will have on the hammer. In contrast, more complex tools can have internal degrees of freedom, such as a glass of water in which the motion of the fluid (the internal degree of freedom) is not fully determined by the current position and orientation of the glass. Such objects can be difficult to control. Here we use a robotic interface to simulate complex objects with internal degrees of freedom and find that subjects are able to learn to control the objects and that the pattern of movement found across subjects is similar. We develop an optimal feedback control model and explain complex object interactions as a simple trade-off between effort and task accuracy.

Humans regularly interact with objects with internal degrees of freedom from carrying a glass of water to using a cloth to polish a table. While objects with no internal degrees of freedom can be regarded as a fixed extension of our limbs [1] , [2] non-rigid objects pose a more complex control problem. The state of the object can often only be influenced indirectly and with a significant time delay, and requires the acquisition of an internal model of the object's dynamics [3] , [4] . We are relatively experienced with simple objects of this class such as carrying a cup of coffee. In contrast, more complex objects with internal degrees of freedom can be highly counterintuitive to manipulate and it may take a long time to learn how to control them, for example when using a lasso. Recently, stochastic optimal feedback control has emerged as a normative framework for human motor coordination [5] – [7] . Given the dynamics and noise characteristics of our limbs, an optimal behavior can be computed that optimizes certain criteria such as a trade-off between effort and positional accuracy. Optimal control theory has been used to explain average movement trajectories as well as trial-by-trial variability in a wide range of motor behaviors, such as obstacle avoidance [8] and bimanual coordination [9] . General principles of human movements have since emerged from this framework such as the minimum-intervention principle [7] . However, the interaction with objects with internal degrees of freedom has not been investigated. We conducted a set of experiments, in which subjects had to manipulate objects with complex and unusual dynamics that were non-intuitive to control. We extended the optimal control framework to such object manipulation with internal degrees of freedom, in which both the position of the hand and the object need to be controlled. Unlike hand-held rigid objects, for which there is a one-to-one correspondence between the state of the hand and the object, for objects with internal degrees of freedom this is no longer true. In addition to the standard optimal control model, in which the hand is modeled simply as a point mass, we also used a more realistic optimal control model, which included the dynamics of a two-link arm [10] – [12] . We show that the trajectories and velocity profiles we observed experimentally could be explained by a simple cost function and that the more realistic optimal control model captures aspects of the data that the point-mass model cannot explain. Furthermore, we tested our model on data from a previous study [13] and show that our optimal control model can also account for the experimental data of a relatively simple mass-spring object.

Results

Subjects manipulated 6 different objects with internal degrees of freedom that were simulated in a virtual-reality setup. Subjects held the handle of a robotic interface that was free to move in the horizontal plane. Their hand was attached to a virtual mass (the object) by a spring (Figure 1). The hand, mass and spring were displayed and the task required subjects to move both their hand and the mass from a start position to a target region within a certain time limit. The robotic interface was used to simulate complex dynamics of the hand-object interaction. The position of the object was updated based on the hand's position according to where , and are the mass, damping and spring matrices (2×2) respectively. For a standard physical system these would be diagonal matrices but to examine the learning of complex objects we included off-diagonal terms in addition to the standard diagonal terms (see Methods for details).

PPT PowerPoint slide

PowerPoint slide PNG larger image

larger image TIFF original image Download: Figure 1. Schematic of the task. Subjects were required to move both their hand (represented by the blue circle) and object (represented by the yellow circle) to a target (green circle) within a time window which was reduced to 0.8−1.2 s over the course of the experiment. The slack length of the spring (blue line) was zero and subjects started each trial with both their hand and the object at the same position within the start region (small and large grey circles represent the initial hand and object positions). During the trial the hand and object position differed substantially due to the complex dynamics of the mass-spring-damper system. To finish a trial successfully both the hand and object had to be on the target with their speed below a threshold of 0.1 ms−1 (low damping conditions) and 0.02 ms−1 (high damping conditions). https://doi.org/10.1371/journal.pcbi.1000419.g001

The objects were non-intuitive to control and subjects had no prior experience of their dynamics. For example, the inclusion of off-diagonal terms in (condition B) meant that the object experienced an additional force that was orthogonal to its movement direction and proportional to its speed. This is a similar field to the standard velocity-dependent curl field [14]–[16] but with the forces applied to the object rather than to the hand directly. Inclusion of off-diagonal terms in (condition K) meant that the spring also applied forces orthogonally to its stretch direction and the forces scaled in proportion to the stretch. Inclusion of off-diagonal terms in (condition M) meant that forces applied to the object caused an acceleration of the object orthogonal to the force. These three object characteristics were all paired with either low damping (conditions B-low, K-low & M-low with small diagonal terms in ) or with high damping (conditions B-high, K-high & M-high large diagonal terms in ).

Subjects first received a training session with a progressively stricter time criterion to facilitate learning and to familiarize them with the dynamics of the objects. To assess stable performance subjects had to continue the test session until they reached a performance criterion of 25% trials achieved within a time constraint. All six subjects achieved this for all objects although some took more trials than others (Table 1).

Figure 2 shows the hand (A: red lines) and object (B: blue lines) paths for one of the objects (condition B-low). Here the hand path is complex and deviated substantially from the straight-line movements characteristically observed during free reaching [17],[18]. In this condition the hand path even shows a loop mid-movement. Moreover, rather than the normal bell-shape velocity profiles of reaching movements we observed biphasic and triphasic velocity profiles in the x- and y-direction respectively (Figure 2A). In contrast the motion of the object (Figure 2B) showed a slightly curved movement with a more bell-shaped velocity profile.

PPT PowerPoint slide

PowerPoint slide PNG larger image

larger image TIFF original image Download: Figure 2. Actual and simulated hand and object paths and velocities for condition B-low. The actual and simulated hand and object paths and velocities for condition B-low in which off-diagonal terms in the viscosity matrix were paired with a low-damped spring. A. The hand path and x- and y-velocities (red lines) with 1 s.e.m. across subjects (standard error ellipse for path plots). The left and right columns show the fits (black lines) of the linear, point-mass optimal control model, and the nonlinear, two-link arm optimal control model, respectively. For the paths (variance explained by the model), the mean movement duration (which was used in the optimal control simulations) and its standard deviation is shown. B. as in A, but for the object motion (blue lines). https://doi.org/10.1371/journal.pcbi.1000419.g002

Similar complex hand paths were produced across the other five conditions (Figure 3). A variety of paths were observed: Subjects made S-shaped (condition M-low), mirror S-shaped (condition K-low) and looped paths (condition B-low) in the low damping conditions that required them to decelerate the mass actively. In contrast, the subjects' hands overshot the target in the high damping conditions, veering to the left (condition K-high), to the right (condition B-high) and passing through the target (condition M-high). A way of conceptualizing the high damping conditions is to imagine dragging the object through a very viscous fluid, hence accelerating the mass as much as possible initially becomes crucial for subjects to finish a trial successfully. We also found that in most conditions the object path was not straight but slightly curved (Figure 4). In conditions K-low and K-high, we observed a slight rightward curvature, while in conditions M-low and M-high the object path was substantially curved to the left.

We compared the subjects' performance to the predictions of an optimal feedback controller. The optimal control law can be computed given the dynamics of the system under control and a cost function on the movement. We extended two existing optimal feedback control models (see Methods for details): a Linear-Quadratic-Gaussian controller with signal-dependent noise, in which the hand was modeled as a simple point-mass [5], and a non-linear optimal control model, which included the full dynamics of a two-link arm [10]–[12]. We modified these two optimal control models to include the dynamics of the different mass-spring-damper objects in the state update equations, assuming that by the end of learning the optimal controller had full knowledge of the dynamics of the objects under control. We constructed a cost function with 5 terms: effort and final positional and velocity errors of the hand and object. This cost function has 4 parameters that change the relative weighting of the different cost terms and based on previous studies we constrained the cost to have only two adjustable parameters. These were fit to the data and fixed to be the same for all object conditions (see Methods). Introduction of a sensorimotor delay did not change the results appreciably (see Text S1 and Figure S1).

Figure 2 shows the model fits (black lines) of the linear, point-mass optimal control model (on the left) and the non-linear, two-link arm optimal control model (on the right) for object condition B-low. Both models capture the salient features of the experimental data and show bi- and triphasic velocity profiles for the hand motion, a more bell-shaped velocity profile for the object motion and the loop mid-movement in the hand path. The model explains 96% (linear model) and 99% (non-linear model) of the variance of the experimental data.

Similarly, for the remaining conditions (Figure 3) both optimal control models yield good quantitative fits to the experimental data and overall explain 79±16% (Figure 3A: linear model) and 84±14% (Figure 3B: non-linear model) of the variance across the conditions and subjects. The prominent characteristics of the subjects' behavior such as the S-shaped paths and overshoots are all captured by the two models. The non-linear, two-link arm optimal control model provides slightly better fits to the experimental data than the linear, point-mass optimal control model. Movement in the x-direction is usually less pronounced due to geometrical constraints and higher control costs in the non-linear model. For example, in condition M-high the linear model substantially overestimates movement in the x-direction, whereas the non-linear model captures the overshoot behind the target seen in the subjects' behavior. Features of the subjects' movements such as the curvature in the hand path in condition B-high and the asymmetry of the S-shaped path in condition M-low are also captured by the non-linear optimal control model. Note that the loop in the path in condition K-low is not accounted for by either optimal control model. Three subjects made loop-like movements whereas the others completed the task with the mirrored S-shaped path predicted by both optimal control models (see Discussion).

Both optimal control models fit the experimental object paths well (Figure 4A: linear model, B: non-linear model) with near straight predicted object paths. However, the non-linear model also shows the correct curvature of the object path across all conditions. For example in conditions K-low, M-low and M-high the linear model predicts a near-straight object path, whereas the non-linear model captures the rightward and the leftward curvature of the object paths. Both are the result of less pronounced hand movement in the x-direction in the non-linear optimal control model due to geometrical constraints and higher control costs as discussed above.

Some of the variance of the hand position signal over time is already explained by the fact that the movement starts at the start point and ends at the target. To provide a null-model for the we used non-adaptive versions of the two optimal control models, that is one of the hand alone, moving from the start position to the target in the same time as subjects in the experiment. We computed between the optimal control predictions and the experimental data as before (see of the non-adaptive controller in Table 2 for the linear and Table 3 for the non-linear model). Note that in principle as defined in the Data Analysis (see Methods) can be negative if the predictions are very different from the experimental data. All of the non-adaptive controllers are substantially lower than those of the optimal control models including object dynamics. For the linear model were from 0.41 (Condition M-low) to 0.84 (Condition M-high) lower. For the non-linear model they were from 0.4 (Condition M-low) to 1.03 (Condition M-high) lower. The for the low damping conditions are slightly higher than those in the high damping conditions because the movement in the y-direction is fairly similar to the experimental data. The for the high damping conditions are very low as the subjects' movements are very fast in the y-direction and the target is being overshot. In conclusion, the optimal control models with object dynamics included are substantially better at explaining the variance of the experimental data.

To analyze the performance of individual subjects on the task, the optimal control analysis was repeated on a subject-by-subject basis for both models (see Methods for details). Table 2 shows the of this new analysis for the linear model and Table 3 for the non-linear model respectively. Figures S2, S3, S4, S5, S6, and S7 show the hand paths overlaid with the optimal control predictions for condition B-low to condition M-high respectively. What becomes apparent from this analysis is that some subjects, on average, performed closer to the optimal control predictions than others (e.g. subject 2 vs subject 6). In addition, although the hand paths of subjects are overall very similar, in some conditions one subject performed slightly differently from the rest (e.g. subject 1 in conditon B-high, subject 3 in condition B-high and subject 1 in condition M-high). It is these conditions that have the lowest and the individual subject fits slightly improve the model fits for these conditions suggesting that between-subject differences in and partially account for the lower . Overall this analysis does not change the main result. The individual subject fits are, on average across conditions, as least as good as those fitted to the mean trajectory (see Tables 2 and 3). Furthermore, the non-linear model still fits the data slightly better than the linear model.

We performed a sensitivity analysis of both optimal control models to the particular weighting used in the cost function. Varying the velocity, effort and object weight over a ten fold range (either smaller or larger) had little effect on the percentage of variance explained (see Text S1 and Figures S8, S9, S10, S11, S12, and S13 for details).

To assess learning across the course of the first 600 trials to the 0° target alone (that was the minimum that all subjects were required to perform), we analyzed the movement trial duration averaged across batches of 20 trials (Figure 5A). With time subjects became faster and adapted to the progressively stricter time criterion that was imposed during the experimental sessions. A paired t-test shows a significant difference between the movement duration of the first and the last batch of the experiment .

PPT PowerPoint slide

PowerPoint slide PNG larger image

larger image TIFF original image Download: Figure 5. Learning object dynamics. A. Mean movement duration across subjects and conditions as a function of batches of 20 trials. Shaded area shows one s.e.m. across subjects. B. Mean Euclidian distance between subjects' hand trajectories averaged across subjects and conditions. Shaded area shows one s.e.m. across subjects. C. as in B, but for the object trajectory. https://doi.org/10.1371/journal.pcbi.1000419.g005

We also analyzed the way that subjects' trajectories changed throughout learning. In particular we wished to assess whether subjects all showed similar patterns of adaptation at intermediate stages of learning that is whether their trajectories were similar during the learning process. To assess this we developed a measure of the between-subject variability. For each batch of 20 trials we computed the average Euclidian distance between the hand and object trajectories (averaged over the batch) of all pairs of subjects. Low values indicate that all subjects produce similar trajectories and high values represent dissimilar trajectories (see Methods). Subjects started out with very different hand and object trajectories at the beginning of the experiment and their behavior became more similar as they improved their performance (Figure 5B and 5C). A paired t-test between the first and the last batch of the experiment shows a significant difference in the variability measure (hand trajectory: ; object trajectory: suggesting that the improved performance was due to subjects converging to similar trajectories. Therefore although the final movements were similar across subjects the pattern of trajectory change during learning was idiosyncratic. Analysis of individual conditions (see Figure S14) shows that this is true for all conditions except K-low and K-high. In condition K-low, three subjects made looped movements rather than the inverted S-shape performed by the remaining subjects. The fact that not all subjects converged to the same solution is reflected in the slightly increasing variability measure. Similarly in condition K-high, subject 3 (see Figure S5) performed the task differently from the rest. When subject 3 is removed from the analysis (see Figure S14), convergence of behavior for the remaining subjects is found as before.

Even after the long exposure that subjects had to the dynamics of the objects during the experiment, they failed to finish the task successfully within the time limit in some trials. Besides the effects of motor noise, another factor that might have contributed is that subjects did not fully learn the objects' dynamics and that a residual uncertainty about the internal model remained. To investigate the effects of model uncertainty and incomplete learning, we adapted the linear optimal control model above in accordance with [19]. In Izawa et al. 2008 incomplete learning was modelled as uncertainty with regard to the internal model. The internal model was represented as . The parameter expresses incomplete learning, e.g. indicates that only 50% of the internal model is learnt. The parameter is a Gaussian random variable with standard deviation that captures the uncertainty about the internal model. We adapted the linear optimal control model to include model uncertainty ( ) and incomplete learning ( ), which are the same values used by Izawa et al. (see Text S1 for details). The of the simulations are shown in Table 2 and the hand and object path in Figure 6. The model fits are considerably better than for the linear model without model uncertainty (an average of 88% versus 79%) suggesting that subjects remained to some degree uncertain how to predict the consequences of their actions on the object and that they adjusted their movement strategy accordingly.

PPT PowerPoint slide

PowerPoint slide PNG larger image

larger image TIFF original image Download: Figure 6. Predictions of the linear optimal control model with model uncertainty and incomplete learning. A. The actual and simulated hand paths for all 6 conditions (in the same format as Figure 2). The fits (black lines) of the linear, point-mass optimal control model with model uncertainty and incomplete learning. B. as in A, but for the object motion (blue lines). https://doi.org/10.1371/journal.pcbi.1000419.g006

We also tested our model on previous experimental data [13] of a relatively simple mass-spring object. The task was to transport a mass-on-a-spring to a target (for details see [13]). Effects of varying movement distance, movement duration and resonant frequency of a simple mass-spring object on human object manipulation were examined. However, in contrast to the current study, there were no off-diagonal terms in the object dynamics. Dingwell et al. used a smoothness criterion on the object path to explain their data. In their study they set out to model the effects of changing the movement distance and movement duration for a fixed resonant frequency of the spring (Experiment B). Our point-mass optimal control model predicts a transition from approximately uniphasic velocity profiles of the hand for slow movements to triphasic velocity profiles for faster movements. In addition, the model predicts that the effect of increasing the movement distance results in scaling the velocity profiles without changing its shape. Both these predictions were observed experimentally (Figure S15 shows experimental data with optimal control model predictions). In an additional experiment Dingwell et al., examined the influence of the resonant frequency of the spring on the velocity profiles of both object and hand (Experiment C). Our point-mass optimal control model yields the same velocity profiles when the resonant frequency is kept constant, independent of how the values of the spring constant and object mass are chosen. High resonant frequencies predict triphasic velocity profiles whereas low resonant frequencies predict uniphasic velocity profiles for the same movement time as the optimal control solution. The optimal control model always yields uniphasic velocity profiles for the object. All these features were observed experimentally (Figure S16 shows experimental data with optimal control model predictions).