“@albertocairo The circle is meant to show a yearly cycle, right?” “The distortion caused by the circular layout makes the timeline of Tunisia looks shorter than that of Yemen, when in fact the period depicted for Yemen is much shorter” “I’m siding with @blprnt rather than @albertocairo in the great circular timeline debate of 2015.” “I’m with the straight line version, if the purpose is clarity. To read the circles I need to perform, like, a zillion eye movements as a I move along the scale.” Sample quotes from Twitter. In addition to the passages above, there were some more heated Tweets as well (all caps and expletives).

Public redesigns that are broadcast to thousands, potentially millions of people around the world are a new phenomenon, a product of the social media era. Such reach and immediacy can have both benefits and disadvantages. On the plus side, it is easy and cheap for redesigns to be widely seen and discussed by the data visualization community. On the other hand, because they take place on social media, such redesigns automatically collapse multiple audiences into a single context. It is a stark departure from traditional design-studio-based peer crits where colleagues share the context and intent of a piece, know the person whose design is being critiqued, and are physically present. Online, public redesigns can feel faceless and less personal, quickly inviting animosity and bitterness.

In the Cairo example, the initial context was pedagogical: his student had asked about radial graphs and Cairo was hoping to illustrate the effectiveness of linear versus radial solutions. Once on Twitter though, the conversation took on a different tone, including questions about aesthetics, Cairo’s authority in the subject matter, the importance (or lack thereof) of the data shown, the role of design, etc. On the one hand, this broadening of topics is something to aspire to during a critique. On the other, the antagonism that sometimes accompanies such exchanges would be better left out of the process altogether. But how to balance these tradeoffs: reach vs. context? And how to optimize so that most of the discussion around a redesign is productive?

Redesign: bridging the two cultures

The field of visualization sits at the intersection of two very different intellectual traditions. On one side of the family, visualization traces its roots to art and graphic design. On the other side, it’s descended from computer graphics and the tradition of scientific experiment. It’s worth taking a step back and describing some of the morés and norms in each field, and how they conflict in the case of visualization criticism.

An essential way that literary and artistic fields move forward intellectually is criticism. Colloquially, “criticism” implies negativity, but of course there’s a much richer tradition in the arts and humanities. As Bardzell puts it, “Speaking generally, criticism refers to an expert of a given domain’s informed exercise of judgment; familiar examples include film and literary critics, architectural criticism, and even a qualified Master of Wine’s ability to judge wine.” [3] One reason to emphasize this definition is to avoid a trivial misunderstanding: an endorsement of “visualization criticism” isn’t a call for general negativity.

In the design community, there’s a particular flavor of criticism that is common: the design critique. Unlike more abstract or philosophical literary criticism, a classic “crit” usually has the goal of improving a work in progress. A key part of the critique process is the social context: the fact that it’s a group of colleagues having a serious, high-level conversation. It’s rare for a crit to be held in public; they’re more commonly found in studios and schools, comprised of professionals and students — people dedicated to their craft, with a shared knowledge of the context for a design, exercising careful judgment.

In science, we see something different. Science seeks to remove human judgment from the process of testing hypotheses. (Judgment certainly survives in funding and publication processes, but that’s a separate story.) A hypothesis isn’t even considered scientific unless it’s falsifiable, that is, there’s a mechanical recipe for showing it’s wrong. The gold standard for a new medical treatment, after all, isn’t the opinion of a committee of doctors, but a double-blind experiment — ideally, an experiment that is replicated successfully by more than one group.

In the case of computer graphics, one of the scientific ancestors to data visualization, we see a series of test images that have become de facto standards. For instance, researchers who work on 3D rendering often start with a particular teapot model. Scholars of image compression and enhancement have for decades used the (notorious) “Lena” Playboy portrait. The goal is to apply new techniques and ideas in a well-understood context, so anyone may perform a direct comparison with previous work.

Comparing these traditions explains some sources of confusion in criticism by redesign. Seen from a tradition where critique largely happens in private, a public redesign can feel unduly personal and exposed. A critique that happens in public, available to anyone who clicks the right link, loses much of the context necessary for thoughtful commentary. Meanwhile, from the point of view of an experimental scientist, the necessary human — and ultimately subjective — professional judgment that is involved in evaluating a redesign may seem like a cheat. We’ve seen evidence of both these reactions from practitioners in the field.

Despite these issues, the method of “critique by redesign” may be just the blend that will help advance the field. The truth is that perceptual psychology and related sciences can provide some guidance for visualization, but are nowhere near advanced enough to completely answer all real-world design questions. Ultimately, human judgment remains essential to the process. And while the lack of context and collegiality can make a public critique problematic, we believe there are “rules of engagement” that can bring out the best in the process and avoid some of the worst pitfalls.

Rules of engagement

Design is not a science. But “not a science” isn’t the same as “completely subjective”. In fact, the critique process has brought discipline to design for centuries. For visualizations which are based on an underlying shared data set, there’s an opportunity for an additional level of rigor: to demonstrate the value of a critique through a redesign based on the same data.

Criticism through redesign may be one of the most powerful tools we have for moving the field of visualization forward. At the same time, it’s not easy, and there are many pitfalls, intellectual, practical, and social. How can we use the tool of criticism to best advantage, with awareness and respect for all involved? Here are some suggestions, which fall into three categories: maintain rigor, respect for designers, and respect for critics.

1. Maintain rigor

As with a scientific experiment, it’s important to know the reason for a redesign — what is being “measured”, in a sense. There are many possible goals for a visualization: to communicate data, to spur conversation, to persuade, to draw in viewers, to create an aesthetic experience. A critic who creates a redesign should be explicit about the goal — and the fact that they may be interested in a different goal than the designer. This is especially important on the web, where people may stumble on the result without knowing any other context.

Second, critics must be honest about any simplifying assumptions. If a redesign shows less data than the original, that should be mentioned up front. Otherwise, there’s a danger that any perceived simplicity of a redesign is really just the result of a reduction in data.

Part of maintaining rigor is acknowledging situations where professional judgments don’t agree, and finding ways to come to an understanding. Sometimes people will look at a side-by-side comparison and come to opposite conclusions. (We saw this in the radial timeline redesigned by Alberto Cairo, for example.) The first step is to have a conversation about the source of the disagreement. Very often it turns out that different professionals have different criteria for success for a visualization, or have different goals in mind; clarifying these is extremely useful to the field. Other times, however, people simply have different intuitions about clarity or legibility. In these situations, it may make sense to turn to a scientific experiment. This should not be viewed as a failure of criticism, but rather a success: a crisp, testable scientific question is a rare commodity.

2. Respect the designer

All redesigns have the potential to seem adversarial, as if the critic is pointing out flaws in the designer personally, asserting their own superior skills, or even, as in Tufte’s redesign, assigning some blame for a disaster. In the age of viral links, the effect is doubled: seeing negative comments about your own work being reblogged, retweeted, and reposted isn’t a pleasant experience.

Making the process more friendly for the designer is good for many reasons. It’s kind, of course. Beyond that, it opens up the possibility of a conversation. Often the original designer of a piece has thought more about its particular problems than anyone else. We recommend bringing the designer into the conversation at every level: talking with them in advance about a public critique, giving them a chance to respond, and at every turn treating them as a competent professional.

Critics can take many steps to soften the emotional impact of redesigns on the designer.

First, if you could make a point by criticizing several different visualizations, pick whichever comes from an organization with the highest prestige. If you have to point out flaws publicly, it’s better to do it to the New York Times than a first-year student. It’s not always true that more established designers have thicker skin, but they ought to. Second, if it makes sense in the context of a critique, note the positive aspects of a design — that is, the features that need no redesign. If the color and typography are excellent and need no change, it might be a good idea to call that out.

Third, to make pedagogical points, consider a “backwards” redesign, one that takes an excellent visualization and makes it worse. In other words, take a visualization that exemplifies the point you’re trying to make (say, good labeling or a good color palette) and then remove key elements (most of the labels, or change colors to vibrate against one another). This may give you the same effect as a traditional redesign, but leave the original designer flattered rather than aggrieved. It’s also a good way to move past the idea that “criticism” implies a negative view of existing work.

3. Respect the critic

Criticism is hard, as hard as design. Indeed, in established media (books, movies, music) good critics are recognized as experts in their own right. As a field, we should give the same respect to our visualization critics. One occasionally hears designers complain that a critic isn’t an active practitioner. Well, that may be true, but it shouldn’t matter.

As a consequence, we shouldn’t let some roughness in a redesign be an excuse to ignore it. A redesign doesn’t need to be perfectly polished to be effective. It may even be better for it to be slightly rough. After all, a redesign isn’t a competition between critic and designer, but instead a proof point for ideas about design.

A final point for designers is to keep in mind the goal of the critique process: ultimately, none of this is a personal evaluation, but instead a way for the field as a whole to improve.

Conclusion

Data visualization is still a new field. It’s already become an essential medium for journalists, scientists, and anyone else who needs to understand data. But the medium is far from understood. It’s early still, and there’s a lot of room for improvement.

We need more criticism, and redesign is an essential part of visualization criticism. But with so much of it happening on the web — in public, instantly in view of everyone involved, available to the world without context or preparation — it can be a difficult process. In this chapter our goal has been to start a conversation about how criticism can be most productive, and cause the least stress, for those involved.

We’d like to end with a thought about the future. The redesigns and criticism we’ve discussed have very much been focused on technique. That’s appropriate for a new medium in its growth phase. Compared to criticism of movies or books, however, it’s unusual. Think of a typical book review: certainly writing style and form figure in reviews. But usually at least as much space, often much more, is given to questions of content: to character, to plot, to atmosphere. We’ll know that visualization has matured as a medium when we see as much criticism about content as we do today about technique.

Acknowledgments

Following our own precepts, we’ve tried to make this essay part of a larger conversation. The impetus came from an online critique by Alberto Cairo, along with discussion that followed on Twitter, blogs, and email. In particular we’d like to thank Moritz Stefaner, Jan Willen Tulp, and others for helpful discussions and pointers.

[1] Visual Explanations: Images and Quantities, Evidence and Narrative.

[2] http://www.thefunctionalart.com/2014/11/redesigning-visualizations.html

[3] J. Bardzell / Interacting with Computers 23 (2011) 604–621

[4] http://en.wikipedia.org/wiki/File:2004_US_elections_map_electoral_votes.png

http://commons.wikimedia.org/wiki/File:Gastner_map_purple_byarea_bycounty.png

http://commons.wikimedia.org/wiki/File:Countycartlinear1280x1024.png