I wish I’d checked out the Mathew Brady collection at the National Archives a lot sooner. For the past few years, I’ve been experimenting with techniques for animating historical photographs, and I’ve had difficulty finding suitable material to experiment on—especially groups of three or more pictures taken at the same time from different perspectives. It turns out that the Brady collection at the National Archives contains thousands of these, all conveniently available both on their own website and at the Internet Archive. As I say, I wish I’d checked it out sooner.

Mathew B. Brady—shown at right in an animation of National Archives item #525437—may well have been the best-known American photographer of the nineteenth century. He and his associates produced the most important visual record of the Civil War, earning him recognition as the father of photojournalism, but he was also responsible for a great deal of general studio portrait work. The National Archives holds thousands of Brady’s glass negative plates, and many of these contain photo matrices: multiple images organized into rows, or columns, or both. Much use has been made of the two-image matrices which are recognizable as stereoviews. However, the matrices containing three or more images haven’t been very thoroughly exploited yet. In this blog post, I’ll describe and demonstrate some cool things we can do with them.

One helpful way of analyzing photo matrices is in terms of the steps by which their images differ from each other. The most important kinds of step for analyzing Brady plates are time steps, horizontal steps, and vertical steps. Time steps are just different moments in time, which I like to identify by number (1, 2, 3, 4, etc.); think of the intervals between frames in a typical strip of motion picture film. I’ll return to these a little later.

The other two kinds of step involve different perspectives, sometimes corresponding to different lenses, and sometimes to realignment of the camera between shots. Horizontal steps, to which I assign letters a, b, c, etc., range from left to right, while vertical steps, to which I assign letters x, y, z, etc., range from top to bottom. In this scheme, the layout of an ordinary stereoview could be expressed as 1ax, 1bx. There are scads of these to be found among the Brady negatives. But we also find combinations such as 1ax, 1bx, 1cx (which I’ll call “triscopic”) and 1ax, 1bx; 1ay, 1by (which I’ll call “tetrascopic”), and it’s these more complicated configurations that I want to focus on here.

The “extra” data in these image matrices might not seem very useful at first glance, but first glances can be deceptive. After all, two views of a single subject from side-by-side vantage points would have seemed equally useless to most observers in the year 1800, but since the 1830s stereoviews have been made and used for stereoscopic viewing: an image from a leftward viewpoint is presented to the left eye, and an image from a rightward viewpoint to the right eye, producing an illusion of three-dimensional depth or “solidity” (στερεός or stereós = “solid”). Stereophonic audio arose half a century later (via telephone, at first) and follows a similar logic, using vibrating membranes in place of lenses and pictures, and substituting left and right ears for left and right eyes. In both cases, the stereo prefix refers in practice to a pair of two stimuli presented to a corresponding pair of two sensory organs, a matter of number which the terms binocular and binaural do a better job of expressing.



But this two-track model isn’t necessarily inherent in stereo as such. The stereotype, in its literal sense, is simply a “solid” printing plate for a page of text, cast from set-up moveable type and used in place of it; another name for the same thing, perhaps coined to mimic the sound of the printing process, is cliché. The stereotype and cliché live on today mostly as metaphors which we don’t remember are metaphors, much like “sidetrack” as a term with origins in railroading. Lest you think I’m getting sidetracked here, my point is this: stereo has come to be associated with pairs of two perspectives because these have historically seemed sufficient for creating an illusion of three-dimensional “solidity” to people with two working eyes and two working ears. Creatures with compound eyes, or Johnston’s organ for detecting sound vibrations, might feel differently.



And so might human beings questing after the moving target of virtual reality, which is always located a step or two beyond whatever counts as ordinary at any given time. Surround sound and other similar strategies are now recognized as delivering something more “solid,” more stereós, than anything that comes out of just two speakers. They accomplish this by recording and playing back sound at more than two points; for example, quadraphonic sound uses four channels routed to four speakers. The effects on auditory perception are complicated, but one consequence is that if you turn your head, the input to your ears can shift with it to provide a more accurate or realistic sense of source localization within space. Similarly, a virtual reality headset can vary the image pairs it presents to your eyes to follow the movement of your head, and Google’s Omnitone can handle ambisonically recorded audio according to the same principle, varying the audio it sends to your two ears. All of which is to say that sound and image data captured from more than two vantage points can feed into heightened forms of sensory display, even though no human has more than two working eyes or two working ears—at least, anatomically speaking.

Back in the 1860s, when Mathew Brady’s photographers took stereoscopic image pairs, the results could be profitably printed, sold, and enjoyed as stereoviews. There wasn’t any technology available at the time for making similar use of triscopic or tetrascopic image matrices per se, but these came into being for other reasons. A tetrascopic matrix could have been split horizontally into two fully separate stereoviews, and a triscopic matrix could have been split vertically into two stereoviews with one frame shared between them. However, the goal of Brady photographers usually seems to have been just to secure multiple single-image negatives in the interests of efficiency, especially when it came to commercial portrait photography: prints could be made from all the images on a given plate at once, and then cut apart and individually mounted. In effect, the “extra” data about spatial relationships was captured accidentally, and it survives today only by chance. But that doesn’t mean we can’t make good use of it.

Let’s consider one relatively simple way of doing so. The illusion of the “moving picture” most often involves showing photos in the order in which they were taken—that is, in order by time step. But not always. The so-called “frozen moment” effect is created by simultaneously triggering a whole group of cameras placed along a path in space and then showing the images in order by camera position. This results in an illusion where the spectator has the sensation of moving in space along the camera path while time stands still. Many historical image matrices created for entirely different purposes lend themselves to similar treatment, with the path often forming a closed loop of some kind.

The simplest case is the stereoview (1ax, 1bx), where we can loop between horizontal steps a and b to create what’s known as a “wiggle GIF” as an alternative to actual stereoscopic viewing. Here the path bounces back and forth between two points, giving the spectator the impression that his or her head is being rattled from side to side. Many two-frame wiggle GIFs have been created from Brady plates by many different people, and I’m not going to duplicate or go further into those efforts here. Indeed, I’m not a fan of the two-frame wiggle GIF in general, and I’ve written here in the past about my attempts to make animated stereoviews less jarring by interpolating intermediate frames (“tweening”), either through simple cross-fading or through morphing.

But adding more source images into the mix has much the same effect as tweening, and a matrix of the form 1ax, 1bx; 1ay, 1by—which I’m calling a tetrascopic matrix—allows us to trace a more complex circular path. If we loop the frames in the order 1ax, 1bx, 1by, 1ay, the viewer will experience the sensation of a looped movement left, down, right, up. Allow me to demonstrate. Two tetrascopic Brady plates, #526367 and #526373, depict a crowded street scene in front of a church. These didn’t attract much attention until 2014, when the Washington Post reported on Paul Taylor’s theory that they were taken during Abraham Lincoln’s funeral procession past Grace Church in New York City on April 25, 1865, from a window in Brady’s studio across the street. The church is definitely Grace Church, although the specific date and event remain debatable. A prolific creator of wiggle GIFs known as Chubachus has already created an animation from two of the four frames available on #526373, as though it were an ordinary two-frame stereoview. But my own animation of all four frames from both plates is shown below.

We might need to zoom in to get the full effect.

Even though these images were taken more or less simultaneously, the exposure times don’t appear to have been identical. In #526367, the third (lower right) frame shows unique traces of double exposure across a limited area at lower center left, which I’m not sure how to account for; it seems odd that just this one part of the crowd would all have moved in unison, so maybe there was something wrong with the lens. But both frames three and four also appear to have been exposed slightly longer, or at least over a slightly different period, than frames one and two. The most compelling evidence of this is the blurred head of a moving child seen at the center of the frame animated at right, which seems to have moved further while frames three and four were being exposed than it did while frames one and two were being exposed. At least, I think that’s what’s going on here; can anyone offer a better explanation?

The vast majority of tetrascopic Brady plates are studio portraits, of which there are hundreds upon hundreds. However, I’ve seen only one single four-frame animation based on one of these plates before: a portrait of Samuel Clemens and another person, possibly General Thomas Williams, which was posted by the National Archives on November 30, 2015, in honor of Clemens’s 180th birthday. I really don’t understand why there aren’t more of these out there—or why they’ve eluded my persistent keyword searching if they do exist.

Below is a little gallery of four-frame animations I’ve created myself from Brady tetrascopic portrait plates, with each presented in a sequence clockwise from the upper left frame, although this doesn’t always result in movement in the same direction. First, here are #526900, General Louis Douglass Watkins and his wife Mary E. (Rousseau) Watkins; #525352, Representative James Tracy Hale of Pennsylvania; and #525345, Annie Surratt, whose mother Mary Surratt was hanged as a co-conspirator in the assassination of Abraham Lincoln.

Next, here’s #529356, General Thomas Francis Meagher, leader of the Irish Brigade; #526396, General Charles Ewing with “Major Commagee” (should this be “Commager”?); and #526954, identified simply as “Boy.”

And finally, here’s #526081, General George Armstrong Custer; #529144, General Alfred Thomas Archimedes Torbet and company; and #527139, G. W. Farwell—I’m not sure who he was, but his over-the-shoulder portrait stands out as unusual and interesting.

Another kind of image matrix that turns up repeatedly among the Brady negatives is triscopic: 1ax, 1bx, 1cx. These plates would have been taken by a camera with three lenses mounted in a row side by side. If a tetrascopic matrix is analogous to quadraphonic sound, a triscopic matrix is analogous to a combination of left, right, and center audio channels. The best loop sequence for animating a triscopic matrix seems to be 1ax, 1bx, 1cx, 1bx; or, in other words, pivoting back and forth from the middle frame in alternating directions. The resulting path is essentially the same as that of a two-frame wiggle GIF, but with double the number of frames (one being repeated twice per loop) and one and a half times the data. Many triscopic Brady plates have been animated in the past as two-frame wiggle GIFs, but below are some four-frame animations I’ve made that take advantage of all three images in the way I’ve just described. The source plates (#529433, #529441, and #529448) are in better-than-average condition, but they’re typical in their stability of movement.

So far, we’ve been dealing with groups of photographs taken simultaneously through multiple lenses. An example at the other end of the spectrum is #526102, a “camp scene.” This is a another tetrascopic matrix, but in this case the images were taken not just from different perspectives, but also at different moments. I believe the chronological sequence runs clockwise from upper left, judging from continuities and changes among the images, with a few of the most revealing bits of evidence shown below. It’s definitely either this order or the same order reversed, and my assumption is that bystanders were more likely to move into the frame, attracted by the novelty of the occasion, than knowingly to move out of it.

Here’s an animation that loops through the four time steps (1ax, 2bx, 3by, 4ay), with one of the soldier’s heads used as a fixed reference point for alignment.

If we zoom in on the center of the scene, we can see lots of wonderful detail.

I’m not the first person who’s tried to bring Brady plates of this sort to life as animations. Chubachus has created a whole YouTube playlist of “Film-Like Civil War Animations” created mostly from Brady plates like this one, including one of the very same scene. But his animation of #526102 displays the frames in a different order: 1ax, 2bx, 4ay, 3by, according to my analysis. I’m pretty sure this isn’t the right chronological sequence—more than one person ends up moving to a new position in frame three and then back again in frame four—but the spatial effect is also different, presenting a rapid side-to-side jitter rather than a circular motion. Four of the “film-like” animations of Brady plates on Chubachus’s playlist do display a circular motion—as does at least one other, not on the playlist—but he treats thirteen of them like #526102, and as far as I’m aware he’s never animated more than two simultaneously exposed frames under any circumstances. The circular motion in space may only be a happy accident in the handful of cases where it’s present.



My own feeling is that a circular motion should always work better for bringing out the perspectival richness of tetrascopic matrices through animation. The question is whether this approach will also be consistent with chronological order when the images weren’t all taken at the same time. A close study of #526102 shows that its frames were exposed in a clockwise sequence that conveniently follows a circular path of perspectives. But is it typical of its kind? I think it probably is. It’s hard to know just what kind of equipment Brady’s photographers were using, but one multiplying camera designed by Antoine Claudet in the early 1850s, described here, had a back that could be made to slide both vertically and horizontally, with the vertical and horizontal movement controlled by two different adjustable racks. With such a camera, the most efficient way to expose four frames in succession should have been to move the plate in a circle (e.g., top left, top right, bottom right, bottom left), adjusting just one of the two racks for each exposure, rather than to “retrace” to one side of the plate with a double adjustment halfway through (e.g., top left to top right, then bottom left to bottom right). That would explain the pattern we see on #526102. Granted, the photographer in this case wasn’t necessarily using a Claudet-style camera—and I’m not even sure how common they were—but he must have been using some piece of equipment for which a clockwise sequence made mechanical sense.

That isn’t to say that every similar-looking plate will turn out to be structured identically to #526102. The movement I’ve described could have started at any of the four frames without affecting the efficiency of a Claudet-style camera, and we do seem to find some variation on this front among Brady plates. For example, #529323 appears to run clockwise beginning with the lower right frame instead of the upper left frame—again, assuming that people were more likely to move into the frame than out of it. But when we loop the images, it doesn’t really matter where the sequence begins and ends. The movement could also have run counterclockwise rather than clockwise, although if I’d been the photographer, I think I’d have tried to be consistent about this so as to avoid mistakes. In any case, if we were to get that detail wrong in “playback,” our loop would just run backwards in time.

One thing I have yet to find is a Brady tetrascopic matrix with four separately-taken exposures that were definitely created in non-circular order. That said, figuring out the original frame sequence can be a bit of a logic puzzle. A nice example is #529298.

Also worthy of note, but not shown in this excerpt, is a buggy with two horses in the foreground of frames A and B only. Can you put these four frames into their original order, bearing in mind changes in pose, adjustments of clothing, and figures appearing or disappearing from view? Good luck! The solution I came up with myself will be found at the end of this essay. Meanwhile, we sometimes find so little movement between separately-taken frames that it would probably be futile to try to work out their sequence, as for example with #529309, shown below in a looped clockwise animation.

Some Brady plates look at first glance as though they contain eight images taken simultaneously of the same subject with an eight-lens camera (and such cameras certainly existed). However, closer examination generally reveals that we’re instead dealing with two tetrascopic matrices side by side, taken at different moments, consistent with a four-lens camera with sliding back such as was commonly used in carte de visite work. A typical example is #526629, with eight similar-looking portraits of General Robert H. Milroy. It was exposed twice (time steps 1, 2) with each exposure leaving four images of two horizontal steps (a, b) and two vertical steps (x, y). The first exposure occupies the left-hand half of the plate, while the second exposure occupies the right-hand side of the plate and is rotated about 0.27° counterclockwise relative to the first exposure. (Time steps on Brady plates sometimes come accompanied by a slight rotation, suggesting that the plate was tilted while being moved into place for a new exposure.) The overall form of the matrix is 1ax, 1bx, 2ax, 2bx; 1ay, 1by, 2ay, 2by.



Below are animations of that same plate; #525370, David Edgar Herold (cataloged as “Harold”), accomplice of John Wilkes Booth in the Lincoln assassination; and #526649, General James Lawlor Kiernan. My strategy here is to alternate between four-frame loops of the two exposures: 1ax, 1bx, 1by, 1ay, 2ax, 2bx, 2by, 2ay.

In the examples shown above, the subjects’ movements between exposures are fairly subtle, and General Kiernan might not even have moved at all. The portrait of Mathew Brady I shared at the beginning of this essay is another example of the same type. But the subjects’ movements become more conspicuous, and even comical, in the next group of examples: #525628, Charles Ten Eyck; #527644, General W. Hughes; and #527656, “Gentleman.”

One unusual case is #525956, depicting General Daniel Edgar Sickles and his staff. This is a double tetrascopic plate rotated ninety degrees from the usual orientation.

The cut-off bottom of two frames makes for an unfortunate distraction, but if we zoom in on the subjects’ upper bodies we can still enjoy an intact eight-frame sequence with a larger cast of characters than usual.

The eight-frame double tetrascopic arrangement I’ve just described is the most elaborate type of photo matrix we ever find on a single Brady plate. However, it’s not uncommon to find two separate plates that were taken in rapid succession with subtle enough changes in pose between them to allow us to animate effectively across them. In most such cases, the sequence spans two single tetrascopic plates, which isn’t terribly exciting because these eight-frame sequences aren’t any longer than the ones ordinarily found on double tetrascopic plates (for an example, check out the portraits of Representative James Guthrie of Kentucky on #525448 and #525449). But every now and then we can assemble a longer-than-usual sequence. If a sequence spans one double tetrascopic plate and one single tetrascopic plate, for instance, we can cobble together a total of twelve frames, a prime example being the group of portraits of General George C. Thomas on #529017 and #530140. And sometimes we’re fortunate enough to find a sequence spanning two double tetrascopic plates for a total of sixteen frames. Take the group of portraits of General Henry Walton Wessells (cataloged as “Wessels”) on #526647 and #526648. If we assume a left-to-right order of exposure, I think #526648 must have come first (Wessells tilts his head further towards the camera between shots), and #526647 second (he leaves his head tilted the same way as in the second shot on the previous plate but now leans back in his chair, holding that pose for two shots in a row). My animation of all sixteen frames may be seen at right. A third double tetrascopic plate survives from the same session, #529010, but unfortunately Wessells is posed for it with his body turned in the other direction.

And there are other scenarios worth mentioning besides. Some tetrascopic plates represent a pair of exposures taken at different moments, each of which resulted in a two-image column to produce the overall pattern 1ax, 2bx; 1ay, 2by (see e.g. #525394, with the same subject differently posed; or #525229, with two altogether different subjects). This configuration is equivalent to the middle two columns of a double tetrascopic matrix, and we can find a variety of other “fragments” of double tetrascopic matrices as well. For example, a three-by-two grid typically corresponds to a double tetrascopic matrix minus one of its expected columns (see e.g. #526474), while a single row of four images typically corresponds to the top row of a double tetrascopic matrix, 1ax, 1bx, 2ax, 2bx (see e.g. #528172).

The configuration most liable to cause confusion is a single row of three images corresponding to part of the top row of a double tetrascopic matrix, since this is easy to mistake for a triscopic matrix. A case in point is #527823, a three-frame portrait of Abraham Lincoln. Curiously enough, this is the only Brady plate I’ve seen animated elsewhere using the technique I recommended above for triscopic matrices—and it’s not triscopic!

The older animation loops the three frames in the sequence sequence 1, 2, 3, 2, pivoting around frame 2 as though it were a midway point between frames 1 and 3. However, closer examination reveals that the matrix actually follows the pattern 1bx, 2ax, 2bx. It’s therefore frame 3, and not frame 2, that represents a position partway between the other two frames: it shares horizontal step b with frame 1, and it shares time step 2 with frame 2. For that reason, I’ve looped the frames in the sequence 3, 1, 3, 2, pivoting around frame 3. If we compare frames 1 and 3, it also looks as though Lincoln’s lower body remains stationary between time steps 1 and 2 while his upper body shifts, so I’ve aligned the lower body and allowed the upper body to slouch. I don’t like the resulting animation as much as I do some of the others I’ve shared above, but at least it’s less twitchy than the older animation, and I suppose there might be more widespread interest in an animated GIF of Abraham Lincoln than in one of, say, General Henry W. Wessells.

Below are animations of three more “fragmentary” matrices:

#529500, Major John Johnson Elwell, remembered today mainly for having loved Clara Barton “ all the law allows (& a little more perhaps)” ; matrix of form 1ax, 1bx, 2ax, 2bx ; animated as 1ax, 1bx, 2bx, 2ax .

; matrix of form ; animated as . #525308, “Lady”; matrix of form 1ax, 2bx; 1ay, 2by ; animated as 1ax, 2bx, 2by, 1ay .

; animated as . #527967, General George Webb Morell; matrix of form 1ax, 2bx, 2ax; animated as 1ax, 2ax, 2bx, 2ax.

Sometimes three frames in a row display yet another different type of relationship. If the subject is a still life or a copy of an existing picture, the three frames tend to vary almost only by rotation, implying that the photographer shot the same thing three times in succession while moving the plate but not the position of the lens. One of the most interesting examples of this kind is #526139, which portrays of a vase of flowers. The animation below presents a loop of frames 1, 2, and 3 in their original form on the left, and auto-aligned to compensate for rotation on the right. The aligned version confirms that there’s no significant change in perspective here, despite some gentle jostling of the bouquet itself.

There are other issues to contend with among the Brady negatives beyond those I’ve gone into so far, although their effects have already been visible in examples I’ve shared. Some of the glass plates are broken, and even when they’re not, their edges often fall right in the middle of a matrix column. Many frames have been turned into partial vignettes, with outlying parts of the subjects’ clothing obliterated. Others have been intentionally crossed out, or have had notes or numbers scrawled across them. All of these factors can make the work of the animator more difficult and less satisfying.

I’ve limited my scope here to Brady plates held by the National Archives, but there are also a good many similar plates in the Brady-Handy Collection at the Library of Congress. These may be more challenging to analyze and work with, since related frames seem to appear a lot more frequently on physically separate plates. But the two collections plainly complement each other, and it may be possible to reassemble some longer sequences by combining materials from both of them.

What I’ve covered in this post is one fairly simple way of making use of larger Brady image matrices, but I can also imagine a number of other tricks we could try if we want to up our game.

Here’s one. First, we take our double tetrascopic portrait of General W. Hughes and create three morph sequences, each containing three interpolated frames: 1bx → 2bx, 1ay → 2ay, 1by → 2by. In FotoMorph, we can set control points for just one of these sequences and then swap in the other pairs of source images rather than creating three separate morph sequences from scratch. This gives us five time steps for each of the three spatial positions bx, ay, by—e.g., 1bx, 2bx, 3bx, 4bx, 5bx—such that we can loop the sequence 1ax, 2bx, 3by, 4ay, 5ax, 4bx, 3by, 2ay, as seen at right. Now the general’s head appears to turn smoothly back and forth rather than jerking from side to side as before.

And here’s another, although you’ll need a pair of red-cyan glasses to appreciate the results. We take a tetrascopic matrix; create two anaglyphs from it, one for the top row and one for the bottom row; and then loop the two anaglyphs. This display strategy combines true stereoscopic viewing for the horizontal vantage points with a “wiggle GIF” treatment of the vertical vantage points.

We can take a similar approach to triscopic matrices, except that in this case we’re applying both the anaglyphs and the “wiggles” to differences in horizontal perspective.

We can also morph some intermediate frames between the two anaglyphs if we want to produce something less jittery, as illustrated by the animation at right. Nor do we need to stop there, although that’s as far as I’ve yet taken things. We could use a virtual reality headset to view a triscopic matrix so that instead of “wiggling,” the scene would actually adjust to the position of the observer’s head as it moves along an axis from left to right. Throw in breaking developments in DeepStereo, and it might be possible to view a tetrascopic matrix in 3D while adjusting to the horizontal and vertical positions of the observer’s head. The experience would, I think, be something like looking at a hologram.

We have many opportunities today to educe historical inscriptions of various kinds against the grain, presenting their content to our senses in vivid new ways that their creators never imagined—ways that violate their most basic assumptions about what sort of thing they were creating and how people would interact with it.

But few of them offer quite as much fun as this. I hope you’ve enjoyed the Brady animations—look for more of them here in the future!

Solution to the Puzzle: A, B, D, C, or clockwise from upper left. Here’s the picture again for reference: A and B must go together, given the consistent pose of the boys around the post at left, as well as the buggy and horses (not included in the excerpt shown). C and D must go together as well, given the person seen looking out of a window in them. And A, B, and D must also go together because the two young men at the center are mostly missing from C, while the one standing on the right has his hand raised a little higher in B and D than in A. I say “mostly” missing from C because that frame still displays a trace of the bright shirt-front of the man on the right, in a position (underneath the left corner of the windowsill) that implies he was in the act of walking forward away from his previous location while that frame was being exposed. Meanwhile, the strikingly consistent pose of the man at right in frames A and C appears to be a red herring; I think he must have pivoted away from and back to the same pose. The same goes for the placement of the boy with the bright cap and dark trousers at far left; he appears to switch places with the boy next to him at D and to switch back again at C, having buttoned his jacket between B and D.