If you know the basics of Machine Learning you already know how to solve a large body of physics problems without realizing it.

This post is mostly about a tool called Lagrangian Mechanics which lets you solve physical problems like an optimization problem.

In Machine Learning you minimize the loss between the prediction and the label to find the best predictor In Lagrangian Mechanics you minimize the total action of a system to find its motion

Representation Matters

Lagrangian Mechanics has a huge benefit over traditional Newtonian Mechanics in that it’s representation invariant which means you have the freedom to choose the easiest representation to work with.

As an example we’ll discuss the simplest kind of robot, a 2-d pendulum — each of your limbs is a pendulum and your body is made up of a bunch of pendulums.

What we’re interested in this case is describing the motion of the center of the mass at the end of the string so at every time t we need to find 3 variables

p → position of the mass v → velocity of the mass a → acceleration of the mass

v is the rate of change of p over time

a is the rate of change of v over time

So we really just need to predict a single variable p.

p is a sequence of positions of a body from a starting time s to a final time

But how exactly do we go about representing the position of the center of the mass exactly? The obvious first choice is to use Cartesian coordinates x, y.

But now we’re faced with a dilemma — should we center coordinates on the ball itself, the wall, someplace else? Is there any principled way of deciding which coordinate is better?

An alternate representation we could use for the pendulum is:

We can fully characterize all the possible positions of the pendulum

θ → angle between the ceiling and the rope

l → length of the rope

Taken together you just end up with the polar coordinates.

The Polar and Cartesian representations are isomorphic (same mathematically) but polar has more advantages in this case.

l never changes through the evolution of the system so effectively we can describe the motion of this entire system using only 1 variable θ instead of 2 variables x and y.

Configuration spaces

One major reason that the θ representation is better is because if we’re trying from time t to predict the position of the ball at time t + 1 there are only 3 possible solutions

In the Cartesian setting, the solution lies somewhere on the x, y plane but there infinitely many possible directions from time t to t + 1.

But in the Polar setting, the solution can only go in one of two directions.

The circle is the configuration space as in the set of all realizable solutions to the pendulum problem and the specific path the particle takes via this space describes its actual movement and is called the configuration path.

Studying the “shapes” of systems in this manner is part of larger field called Topology. Topology is tremendously useful to study the motion of physical systems so here’s a taste of it.

Let’s take a more complex system, a double pendulum — imagine the first ball as your elbow and you can see how this represents something a bit more interesting, namely your arm.

You can describe this system via two angles

If you fix the first angle then configuration space of the second angle is again a circle so the new space for both angles would look like.

And since there’s infinitely many small circles on top of the larger one the shape we end up with is the famous torus (donut) from topology.

Each point on this 3-d object represents a possible solution to where the double pendulum can be. We’re essentially characterizing the solution of this system as a space.

The actual motion of an example pendulum is then described by a path on this torus.

If you work with Newtonian Mechanics and need to change your representation to see what works best you need to re-derive all your formulas which when you go into the double pendulum realm involves some tricky trigonometry.

The Lagrangian L on the other hand is representation independent which means.

Now let’s define what the Lagrangian is and see how it works.

Lagrangian

Now we know that we can describe the motion of a system using a path but which path will a system actually take in its configuration space?

Nature is lazy

All physical processes take the path that minimizes total action.

A famous example is refraction where the light bends because it’s faster for it to do so.

The Lagrangian of a system L can be understood as its “liveliness”. It’s defined formally as

where T is the kinetic energy and V is the potential energy.

The kinetic energy T is always the same

And the potential energy depends on the type of system being described but for us we’ll mostly be looking at gravitational potential

where

m → mass of object

g → gravitational constant

h → height from ground

The more famous conservation of energy law takes the sum of potential and kinetic energy and it’s been tremendously reliable in pruning out bad physics theories.

What the conservation of energy law tells us is that no new energy can be created and no energy can be lost. Instead we have a trade-off where if T increases then V decreases and if T decreases then V increases.

The total action generated by a system is denoted S and can be calculated by summing up all the Lagrangian of each position the configuration space.

Euler-Lagrange Equation

So now that we can derive the Lagrangian L of any system, how do we get the minimum S and the path that generates it.

The transition from Lagrangian to a solution is done via the Euler-Lagrange equation which will find the path of least action.

The Euler-Lagrange equation is

The formula looks more complicated than it actually is and it’s best understood via a couple of examples.

Falling Ball

Both of my examples are heavily inspired by this series

Suppose we have a ball of mass m falling to the ground

The kinetic energy T is only a function of y because there is no sideways movement

The potential energy V is

So the Lagrangian is

The Euler Lagrange equation will only be a function of y so we can simplify it

Plugging in L

Which simplifies to

So we end up with deriving Newton’s law from scratch 🤯

Moving Cart Pendulum

The last example was to show you how Lagrangian Mechanics works but it’s not an example where Lagrangian Mechanics shines.

Lagrangian Mechanics shines when the system is complex with many variables that would be painfully difficult to model with Newtonian Mechanics.

So we’re going to instead imagine that we have some sort of moving cart with mass M with a dangling pendulum of mass m. My head hurts thinking about the number of different forces I need to model here.

Thankfully Lagrangian makes solving this system far easier since we can just sum up each objects Lagrangian independently.

We can model the position of the cart and ball with 2 variables each for a total of 4 or we can use a trick where we change the coordinate system to make our lives easier.

We make the following 2 substitutions

With this change of variable in place we can derive the Lagrangian L.

Remember, nature doesn't’ care about how we represent it

Kinetic energy is easy. The cart only moves in the x direction and the pendulum can move both in the x and y direction.

Potential energy

So the total Lagrangian is

Once you have L you can just apply the Euler Lagrange equation to see how this system behaves.

Contrast this approach to the Newtonian Mechanics one where

Create a force diagram between each object and every other one to create a set of equations — potentially exponentially many Plug those equations into an ODE solver since most derivatives in physics have no closed form solution — and pray

The difference between Newtonian Mechanics and Lagrangian Mechanics is the difference between imperative and declarative programming. An efficient declarative paradigm like Lagrangian Mechanics will always be preferable in the same that SQL is preferred over UNIX scripting.

Modeling constraints

A final advantage of Lagrangian mechanics that I want to briefly touch on is that it’s trivial to add constraints to a system.

A constraint is some function that f such that

f could be a constraint on the max velocity or it could be a constraint on how far a joint can bend or it can be a constraint on how soft the collision of two objects should be.

You can have as many constraints as you want

And all you need to is instead of plugging L into the Euler Lagrange equation you would plug in where λ’s are weights for how important each constraint is. This trick is called “Lagrangian Multipliers” and is used extensively in convex optimization.

Code

Even though Lagrangian Mechanics is a powerful idea that lets you heavily simplify implementing physics engines, as far as I know it’s not a mainstream idea yet. If you do know of any physics engines that do use it, please let me know and I’ll describe how it works in a real physics engine in a future blog post.

The first time I saw an full implementation of Lagrangian Mechanics was in the fantastic book — Structure and Interpretation of Classical Mechanics. The implementation here is in LISP which is a great language to learn but in my experience a tough language to use on larger projects.

So instead I’d recommend taking a look at the project https://github.com/MasonProtter/Symbolics.jl which offers a Julia implementation heavily inspired by the SICM implementation.

The README specifically talks about the harmonic oscillator which is just a fancy term for a weight attached to a spring.