Exterior Product

Why Linear Algebra may not be last mathematical formalism of space

Linear Algebra

After working with Linear Algebra for a while, you start to forget that some small stuff doesn’t really make sense.

Determinant

The zigzaggy pattern is at first confusing but you’re told to rest assured. You can think of the determinant of a matrix A to be equivalent to the area spanned by its vectors.

OK so what about determinant of the below matrix?

Turns out, determinants are only defined for square matrices. 🤔

OK fine, but if you have the area of 2 matrices A and B then surely you can add them up!

Nope and turns out no-one even knows if the lhs is a function of the rhs.

Cross Product

Geometrically the cross product refers to the area spanned by the parallelogram of the two vectors a and b.

But for some mysterious reason, the cross product is only defined on ℝ³. 🤔

Consider that when you work with any sort of dataset you’re dealing with dimensions much larger than ℝ³, so why is that you see inner products all over deep learning code but not so much the cross product even though it does compute something useful.

Honestly, it’s also weird to be waving your fingers around and changing the results of your computations depending on what you picked. If you maintain a physics related codebase you need to make sure all the libraries that you’re using and are using you, have adequate orientation conversion.

What can we do better?

Turns out Linear Algebra only became popular after Maxwell publicized his famous equations but there were others like Grassman and Clifford that were working with lesser known formalisms which make certain spatial operations much simpler.

While studying the details of Geometric Algebra may seem esoteric right now, consider that for the foreseeable future we’re going to be working with deep models and build robots so it’s only natural that our mathematical tools evolve to better suit us.

Exterior Product

To get a taste of the kinds of things you can do with Geometric Algebra, we’ll be looking at the exterior product.

The exterior product is defined as a ∧ b in some vector space V where a, b ∈ V.

It needs to fulfill 2 properties

1. Antisymmetry: a ∧ b =-b ∧ a

2. Bilinearity: a ∧ (b + λ c) = a ∧ b + λ a ∧ c

So far these 2 properties seem abstract but let’s see an example where Antisymmetry and Bilinearity hold. Geometry!

Determinant

Let’s take a parallelogram with side vectors a and b.

Q:How does the area of the parallelogram change if we double b? A: Area(a, 2b) = 2 Area(a,b) Q: How does the area of the parallelogram change if we multiply b by -2? A: Area(a, -2b) = Area(a,2b) = 2 Area(a,b) Q: How does the area of the parallelogram change if we add c to b? A: Area(a, b + c) = Area(a, b) + Area(a, c)

Areas also have an orientation which we typically neglect in Linear Algebra but is useful for problems like collision detection and culling in computer graphics.

Area(a,b) is the area rotated away from the screen and Area(b,a) is the area rotated towards the screen and their relationship is Area(a,b) =-Area(b,a)

We can clean up the examples into 3 laws

1. Area(a,b) = -Area(b,a)

2. Area(λ a, b) = λ Area(a,b)

3. Area(a, b + c) = Area(a,b) + Area(a,c)

Now if we rename Area(a,b) to a ∧ b you’ll see how the Area of a parallelogram is just a special application of the wedge product

a ∧ b =-b ∧ a

2. (λ a) ∧ b = λ (a ∧ b)

3. a ∧ (b + c) = a ∧ b + a ∧ c

In Linear Algebra we typically think of vectors or matrices as containers holding elements of ℝ, so an n dimensional vector holds n elements of ℝ. Instead we could think of an n dimensional vector as a single element of ℝ^n and perform our computations on this higher level structure.

Any vector can be expressed in terms of the basis vectors . In a 2 dimensional space we have two basis vectors:

And we can express a and b in terms of their basis vectors

The wedge product ∧ has an important property

Now if we take a ∧ b

We get back the determinant formula! And at no point did we constrain ourselves to square matrices.

Cross Product

The Cross Product is usually defined over ℝ³ which means we need to work over 3 basis vectors

We’d like to work with 2 vectors a and b.

If we take their wedge product ∧

If we look at the coefficients they look identical to those of the typical cross product.

And there is no reason for this definition to only apply to ℝ³, we could generalize this easily to ℝ⁴ and beyond.

But instead of using i, j, k as basis vectors we use e ∧ e which is a higher dimensional space. This is a general theme in Geometric Algebra.

In Linear Algebra we typically think of vectors as matrices as containers holding elements of ℝ, so an n dimensional vector holds n elements of ℝ. Instead we could think of an n dimensional vector as a single element of ℝ^n and perform our computations on this higher level structure.

Next steps

Geometric Algebra is a mathematical language to program geometry problems.

It’s extremely likely we’ll be doing more geometry in the future because of how popular deep learning and robotics have become. Why settle on a popular formalism like Linear Algebra if it doesn’t express geometric problems as naturally as Geometric Algebra.

I only discussed a single operator, the wedge product ∧ — there are still many more applications of it and a good way to get exposed to them is to check out the below references.

Acknowledgements

I never thought I’d end up researching something like Geometric Algebra but Adam Nemecek insisted numerous times that it would be worth my while and he was right. He maintains a fun and active Discord community of graphics researchers and mathematicians that discuss how to use Geometric Algebra more broadly. I’d also like to thank u/muntoo for carefully proofreading this article.