Aerial Imaging: I Fly with My Little Eye Digital Imaging Finds a New Home in Small Unmanned Aircraft Systems

An electrical lineswoman hunches against the storm, squinting at a tablet screen and fidgeting nervously. It’s November in northern Ontario, the sleet is blowing 75 kph sideways, and there’s a problem with the power lines. She watches intently for minutes that to her seem much longer…then exhales pent-up tension and breaks into a smile. Special thanks to Aeryon Labs in Waterloo, Ontario, for insight and imagery.

The images on her tablet tell the story—the closeup of the high voltage line on top of the transmission tower (across the river 500 m away) looks intact. A thermal view shows none of the heat spikes that would indicate insulator damage. The problem isn’t up there—she’s not going to have to climb the tower in this weather. Thank you, tiny flying robot.

Views like this are not possible by any other means. Credit: Aeryon Labs

The Value of Perspective

The point of having flying robots, aka small unmanned aircraft systems (sUASs), is as much about imaging as about flying. Amazon Prime Air may someday deliver instant shopping gratification from the skies, but for the near future the most valuable payload a tiny flying robot can carry is usually an image sensor, followed closely by a transmitter to beam images back to the ground for a remote view.

This should be no surprise—the insights we gain from high-level perspectives are profound and incredibly valuable. Whether from trees, hills, towers, or eventually balloons and aircraft, we have always sought higher vantage points. The urge to acquire them is fundamental, built into our bodies (better visibility is one theory for bipedalism) as well as our languages—think of words like “overview,” “supervise,” “surveillance,” and “understanding the big picture.” Poets have written about the gift of “see[ing] ourselves as others see us.” Perhaps the ultimate “perspective photo” is the one astronauts get looking back at earth; those lucky enough to experience it firsthand frequently describe it as life-changing. In some cases aerial views can also be life-saving…or at least day-making.

Perhaps the ultimate “perspective photo” is the one astronauts get looking back at earth. Credit: NASA / Bill Anders, via Wikimedia Commons

Eye in the sky applications

Examples abound of applications eagerly adopting aerial imaging: pipeline inspection, wind turbine inspection, search and rescue, forestry management, mine surveying, precision agriculture (pinpointing application of pesticides, irrigation and fertilizer), construction surveys, broadcast television, sports photography, cinematography, petrochemical flare stack inspection, railway inspection, highway maintenance surveys, bridge inspection, oil rig inspection…the list goes on. In each situation, sUASs provide views that until now have not been practical or even possible by any other means.

Small unmanned aircraft systems can be used for a host of applications, such as wildlife counts in otherwise inaccessible areas. Here, scientists conduct a seal census in the Aleutian Islands. Credit: UAF - University of Alaska Fairbanks

SUASs’ advantages are both clear and game-changing. Small, light, and unmanned, they pose a much smaller risk to people (both pilots and bystanders) and property if anything goes wrong. The minimized mass that allows them to fly also minimizes the kinetic energy of a crash impact. Compared to conventional aircraft, they are much more portable, much more easily deployed—they can be carried in a suitcase or backpack and snapped together without tools—and cost much less to own or operate. They are also more tolerant of poor weather…and operators’ aerial selfie vanity shots.

Petrochemical flare stack inspection—way too dangerous for people, but no problem for flying robots. Credit: Aeryon Labs

Challenges: Motion Pictures from Moving Cameras

Flight capability by battery power is a marvel of engineering in its own right, but image capture brings extra challenges for flying robots. It isn’t as simple as just strapping a “point and shoot” handycam onto the flyer. It’s true that there are rigs that will carry DSLRs and panoramic cameras aloft, but for sUASs, every choice has a tradeoff.

Perhaps most obviously for a flying application, the first issue is mass. Every extra gram affects flight time, range, altitude, balance, stability and manoeuverability. Lifting a DSLR rules out long flights or anything less than perfect conditions. Inexpensive flying robots integrate their own cameras, optimized for small size and mass. They have to depend on off-the-shelf commercial image sensor components, so mobile phone imagers are obvious choices for small size, mass, and low power. Current imagers provide HD resolution video and 14+ megapixel stills in visible spectra or VGA resolution in infrared, but there is more to imaging than resolution. Since flying robots can move rapidly in X, Y, and Z dimensions and must deal with vibration from the motors as well as buffeting winds affecting pitch, roll, and yaw, camera shake is a huge issue. Even with careful design to balance and stabilize camera modules, getting a crisp image in spite of aircraft residual motion is difficult, especially if the image sensor uses a typical rolling shutter. Sensors with top notch global shutters of the type more common in machine vision help to eliminate smear and the “jello video” effect. To further isolate the camera from the flyer motion, high-end flyers build their imager modules into carefully designed custom gimbals that also allow the camera to look one direction while the robot flies in another.

Customized imaging modules with stabilized gimbals allow flyers to offer feaures such as optical zoom, panning, and infrared detection. Credit: Aeryon Labs

Challenges: Resolution and Image Quality

Surveying, agriculture and GIS applications among others demand resolutions higher than flyers can provide in a single shot; as a result, sUASs often take multiple overlapping images that can be processed into simple 2D mosaics. Properly planned and positioned, flyer imagery can supply detailed elevation and 3D models with ground resolutions up to 1cm/pixel.

Photographers of all kinds appreciate remote focus control and optical zooming. Sophisticated flyers provide optical zooms of 20x or more, although this must be traded off against the added mass of lenses and servos, as well as the effect of those parts’ movement on a flyer’s center of gravity. Adding infrared sensors provides new and tremendously valuable levels of insight, but this also multiplies system complexity, mechanically and in terms of data processing and transmission. Almost all flyers now integrate GPS functionality and data—the flyers need it to follow a path home if they lose communication with the remote pilot.

Challenges: Data Transmission

Beyond image capture, sUASs’ next task is to reliably transmit stills and real-time video back to the base station. The first issue is availability of spectrum for communications—commercial spectrum can be quite limited, and while law enforcement and military have their own spectrum, all depend on locale. In RF communications, there is a direct tradeoff between bandwidth, power, and transmission distance—want to transmit a higher data rate from farther away? It will cost you power, which will cost you flight time. As with any bandwidth-limited application, data compression codecs can be very useful—the flyer may choose not to transmit full resolution via real-time image down-link. In this case, there is typically also on-board storage such as with a SD card, so that in the event of packet loss or image degradation otherwise during flight, there is a way of retrieving and physically offloading full resolution images and video immediately post-flight.

Ruggedized tablets provide control and monitoring for some flyers. Credit: Aeryon Labs

With images received, the processing pipeline continues—the host system must interpret and integrate all the data streams into meaningful imagery and show it to operators…preferably on high-performance daylight displays. Smartphone apps serve low-end monitoring, but higher end systems demand larger, more robust displays and finer controls. If you have to pilot a flyer manually, the video feed is vital, although this isn’t the case for more autonomous systems—sUASs such as Aeryon’s Scout and SkyRanger eschew joysticks and follow flight paths swiped across a tablet screen with a Google map-like interface. SUASs are outstanding examples of innovation and the interdependence of hardware and software. The challenges are considerable, but the payback from the unique perspectives they enable is compelling.

Back at the power lines, the tiny flyer has swiftly returned on command, and the lineswoman has quickly broken it down and packed it into its suitcase, ready for its next eye in the sky adventure.