With my design, composition, and data prepared, it’s time to do some rendering! If you recall from my Manson project, this was quite the ordeal last time and cost me a Macbook Pro. This new composition is of similar length but is much more complicated. On the flip side, we’ll only need to render about 1,500 of these videos vs. the 25,000 for Manson. So, am I going to sacrifice my iMac Pro to the cause? Perhaps provision a Mac Pro box on something like MacStadium? Not this time! As I mentioned earlier, Intel was kind enough to send me one of their 18 core systems and I’ve been dying to put it through it’s paces.

Well, as it turns out, After Effects isn’t setup to take advantage of multiple cores by default. While running a test render, I noticed that my CPU usage would plateau around 17% usage. Upon further investigation, my worries were confirmed in the Adobe forums. In the last 3–4 versions of AE, Adobe has not offered a solution for multi processing. Bummer.

Instead of losing hope, I dropped an email to Dataclay. They quickly suggested that I check out the 3rd party After Effects plugin RenderGarden by Mekajiki. RenderGarden promises a setup that gets AE to use all of your cores for rendering. After watching their excellent tutorial videos, it was clear that they achieved this by breaking up larger compositions into pieces and passing each of those pieces to a rendering process. The rendered pieces are then assembled and encoded as needed. Excited, I easily configured RenderGarden and ran a test.

t was faster but not much faster. The issue? RenderGarden was splitting up compositions which were already pretty small (15 seconds) and then putting them back together. What I really need for this use case is the ability to simply render a couple of videos in parallel. They don’t need to be split into pieces since they are already so small. I spent a bit of time thinking through my options but ended up sending RenderGarden an email as well. Their response? It can be done!

If you actually read through the RenderGarden documentation, you’ll learn that they apply two basic functions: seeds and gardeners. A gardener is a process that will do the actual rendering and it’s suggested that you run half as many of these as you have processors. I chose to run eight of these. A seed represents how many times you’d like a video to be split up as part of the assembly process. RenderGarden told me that if I simply set this number to one, their plugin wouldn’t split up anything and simply run eight unique video renders in parallel. Amazing!

Equipped with the knowledge I required, I decided to run a couple of different tests for your enjoyment. First, I would attempt to render 10 and 50 videos using the default After Effects Media Encoder which chooses to render videos one at a time. Then I would try the same quantities with 4, 8, and 16 gardeners! Here are the results:

| | AE Render | 4 Gardeners | 8 Gardeners | 16 Gardeners |

|-----|-----------|-------------|-------------|--------------|

| 10 | 14:07 | 19:16 | 5:10 | 3:26 |

| 50 | 26:29 | 29:00 | 16:00 | 13:30 |

Taking a look at this chart, it’s clear that the benefits of using Render Garden do not come into play until you’re using 8+ gardeners. I decided to use 10–12 gardeners to render the final total of 1185 videos. That should be enough to speed up things significantly. So… how long did it take?

About 12 hours.

Is that fast? I don’t know. The whole process leaves a lot to be desired starting at the beginning with After Effects supporting multi-core computers. Render Forest is a nice solution in the meantime but I would love to see Adobe step up. While the Intel machine had no trouble distributing and handling the job, the real bottle neck starts with composition replication via Dataclay and without multi-core support, the stress on After Effects really compounds. One solution might be to run the 2014 version of AE which was the last to support multi-core processing. I will continue experimenting with all of this software and hardware and publish my findings here.

Thanks

Photo by Brantley Gutierrez

I seem to always be thanking the Foo Fighters and for good reason; they are always willing to experiment with new ideas and have a rich history worth celebrating. Thanks to the band and their organization for allowing me to develop this project. Speaking of history, thanks to setlist.fm for building a community that allows fans to share live music data and then making that data available via an API. If you ask me, this is one of the most emotionally relevant APIs in all of music.

Thanks again to Dataclay for providing the technology that makes this dynamic video generation possible and to their team for supporting all my use cases. Thanks to Mekajiki for taking it upon themselves to build a solution to After Effects lack of multi processing support. You should definitely give RenderGarden a try.

And finally, thanks to Intel for heeding the call of a developer in need. I look forward to further exploring what this 18 core box is capable of and sharing the results here on my blog.