Adventures in Blender’s Video Sequence Editor

Blender’s Video Sequence Editor (or VSE for short) is a small non-linear video editor cozily tucked in to Blender, with the purpose of quickly editing Blender renders. It is ideal for working with rendered output (makes sense) and I’ve used it on many an animation project with confidence. Tube is being edited with VSE, as a 12 minute ‘live’ edit that gets updated with new versions of each shot and render. I’ve been trying out the Python API to streamline the process even further. So… what are the advantages of the Video Sequence Editor. Other than being Free Software, and right there, it turns out there are quite a few:

familiar interface for blender users: follows the same interface conventions for selecting, scrubbing, moving, etc. Makes it very easy to use for even beginning to intermediate users. tracks are super nice: there are a lot of them, and they are *not* restricted: you can put audio, effects, transitions, videos or images on any track. Way to go Blender for not copying the skeuomorphic conventions that makes so many video editors a nightmare in usability. Since Blender splits selection and action, scrubbing vs. selection is never a problem, you scrub with one mouse button, select with the other, and there is never a problem of having to scrub in a tiny target, or selecting when you want to scrub. I’ve never had this ease of use in any other editor. simple ui, not super cluttered with options covers most of the basics of what you would need from a video editor: cutting, transitions, simple grading, transformations, sound, some effects, alpha over, blending modes, etc. has surprisingly advanced features buried in there too: Speed control, Multicam editing, Proxies for offline editing, histograms and waveform views, ‘meta sequences’ which are basically groups of anything (movies , images, transitions , etc) bundled together in one editable strip on the timeline. as in the rest of Blender, everything is keyframable. you can add 3D Scenes as clips (blender calls them strips) making Blender into a ‘live’ title / effects generator for the editor. They can be previewed in openGL, and render out according to the scene settings. it treats image sequences as first class citizens, a must!!! Python scriptable!!!! big feature IMO. (uses the same api as the rest of Blender)

Disadvantages are also present, I should mention a few:

UI is blender centric! so if you are not a blender user, it does not resemble $FAVORITEVIDEOEDITOR at all. Also, you have to expose it in the UI (only a drop down away, but most people don’t even realize it is there) no ‘bin’ of clips, no thumbnail previews on the video files, though waveform previewing is supported. lacks some UI niceties for really fast editing, though that can be fixed with python operators, and also is getting improvements over time. could be faster: we lost frame prefetching in the 2.5 transition, however, it is not much slower than some other editors I’ve used. not a huge amount of codec support: Since Blender is primarily not a video editor, supporting a bajillion codecs is not really a goal. I believe this differs slightly cross platform. bad codec support unfortunately means not only that some codecs don’t work, but that some of the codecs work imperfectly. needs more import/export features (EDL is supported, but afk only one way) some features could use a bit of polish. This is hampered by the fact that this is old code, a bit messy, and not many developers like to work with it.

Needless to say this is all ‘at the time of writing’. Things may improve, or the whole thing gets thrown into the canal 😉

So what have I been up to with Blender’s video editor? Quite a bit! Some of it may end up not-so-useful in the end, but experimentation could yield some refinements. The really good thing about using Python, is that I can ‘rip up’ complex things and rearrange / redo them. So the experiments don’t result in a huge waste. Lets have a peak.

Automatic Updates – take 1- ogler.py:

In the beginning of the project I thought to use Blender’s scene support to create a ‘live’ edit, that did not use video or image clips at all, but instead, referenced actual animation shots directly. This would be done by linking the shots into the edit as scenes, then using Blender’s scene strip support to edit them live as opengl previews. In my more optimistic moments, I imagined pressing ‘render’ directly from the sequence editor, and rendering the entire movie at once.

Early tests were promising, and in fact, for very small (say under 10 shots) this approach could work quite well. However, things were getting slow for Tube, as our shots were a bit too slow for editing live, and I wanted the speed of movie clips without the loss of the dynamic connection. Chris Webber, lead developer of GNU Mediagoblin, swooped to my assistance, and created ogler.py: A sequence strip addon that renders out the linked scenes into opengl previews, and then swaps them in the edit, but can swap them back to scene strips. so you can edit in the scenes, oglify them and edit fast, and then de-oglify to keep the connection live anytime you want.

Sadly, once tube got to around 30-40 shots, this became unworkable. Even loading the file with all the linked scenes would take for ever, and I started to run out of ram. It became clear that the memory requirements for the full movie (around 70 shots) would be too big to just link into one file. Sadly, I oglifyied for the last time, deleting the scene links as I went, leaving burning disconnection and lament, where once was a harmonious connected live edit.

Automatic Updates – take 2- smash_all.py:

After that, updates were not automatic. As shots were animated, I’d open the scene file, open gl preview it (henceforth, the verb is boomsmash) and refresh the edit. Sometimes I’d download a preview directly from an animator, or from helga, and skip the whole boomsmashing thing. With multiple animators working this got a bit tiresome and error prone, and slowly the videos files would drift from ogler’s handy folder, getting scattered on my harddisk, whereever they were dropped initially. Also, we started getting in rendered shots, so these found their way on the edit. These are managed quite well by Helga, as image sequences on the server. I would drop them into the sequencer via the mounted network folder, and they worked surprisingly well – fast, but not realtime. However, I’d every now and then drop a local render in too, complicating things again.

At this points I started to have two problems: One the setup was far too chaotic, and only worked on one workstation. Two, it was slow, and I need realtime performance to do more careful and artistic editing down the line. I decided to solve the organization problem first, as having a working edit is of paramount importance when reviewing and modifying animation shots, for the obvious reason that they have to work together in the edit.

I wrote a simple script called smash_all.py. First time you run it, it boomsmashes the entire list of shots into one folder- and saves the timestamps of the blend files it just rendered. Subsequent runs, it will only do this for changed files (by checking the timestamps), allowing me to run it to automatically updated progressed animation shots. Nice side-effect, all the movie files are consistently named, and live in one folder. The actual process runs for a bit, but, it can be run in the background with no user intervention.

The script is an addon, but currently has hardcoded shot list, and folder locations, it wouldn’t be hard to make a config file it could import instead – in any case, all the paths are at the beginning of the file, easy to edit. We could improve it by getting the user’s home folder from the OS module, making more paths relative to the blend file, and getting a few folders as options. I’d love to hear suggestions, as I’m not sure what the nicest thing would be.

Speed: Sequencer Proxies and proxy_workflow.py:

Blender as I mentioned has a built in ability to do proxies for editing slow footage (examples are hi resolution footage, exrs, footage that lives on a network folder, etc). They are a great way to speed up your edit, and might be a cool way to do an offline edit (I’m trying to see how possible this is with some help from python). Proxies are either movies or image sequences, and can be at 25 to 100% of the resolution of the original- but in jpg or avi jpeg formats, very fast. Each strip has a toggle to enable proxies, and then a bunch of options for desired sizes, timecode, and folder/file locations.

Since the strips I was using were at least partially on the server, I wanted the proxies to be local to my disk for speed, not on the same server with the rendered footage. So I needed the custom folder option, but… setting a custom folder for each and every clip one by one is just not an option for a lazy programmer (more seriously, it isn’t a good option for anybody). Since I plan to explore several workflow enhancements for proxies, I opted to make an single addon (proxy_workflow.py) that contains right now just a single operator, that gets a menu item, and a hotkey (ctrl-shift-a for now). The operator creates a folder ‘Tproxy’ next to the edit.blend file, and populates it with (hopefully unique) subfolders based on the path of each strip. You simply select all the strips you want to proxy, and it will create proxies (but only for movie and image files) in the selection. It also gives you a popup so you can select the sizes required, without having to go strip by strip. Maybe in the future we can make the path an optional entry, for custom workflows.

A known problem: if the same source image or video is used for multiple strips, it will not proxy it, but it will create the custom folder correctly. Simply select one of the strips and do rebuild proxies and timecodes, and all will be well.

Speed: Future work:

It seems that even with local on disk proxies, the edit hits the network for the source files every time you scrub over to a new clip. This means scrubbing within one strip is fast, but the moment you ‘step’ over an edit, you get a little lag, dependent on the network speed. At the Drome, this is almost imperceptible, but at home… the edit literally just hangs. I can think of a few solutions.

bug report/whine to blender coders (or check the code myself) and see if this can be fixed in Blender. No need to check the source if you are using a proxy. temporarily change the source folder to some dummy location, and save the original location in a custom property, and use an operator to toggle back and forth. build the proxies, but then use them *as* the source, and switch back with an operator. do the ‘offline’ part completely outside of blender. just make (low fidelity) copies of everything, local to the edit, outside of blender, and work with that. Export an EDL, or just swap the ‘real thing’ in later. I would prefer to do things within blender, and not have to resort to this.

One would be best I think, unless there’s some reason this lookup to the original file needs to happen that I didn’t think of.

Proxies and Alpha:

This wasn’t done for Tube (it was a midnight hack for a commercial project under deadline) but I ended up using it on Tube. Simply put:

Since proxies are .jpg files they don’t have alpha channels. This can be an issue depending on your edit, if you want a smaller sized proxy of the alpha/image. To solve this I wrote an operator that uses imagemagick to create .tga proxies (bigger, uncompressed, and faster than .png or .exr files) instead, but then renames the extension to .jpg. Blender happily loads them as proxies, but the alpha still works.

Where’s the code?

You can find the code, in its current, unpolished and silly state, on gitorious