"People think of Stitch Fix as this styling algorithm that picks out clothes for you," says Eric Colson, Stitch Fix's chief algorithms officer emeritus. "And sure, we do that. But there are more than 100 data scientists and only six work on that. The rest are doing all kinds of other algorithms."

For example, the company uses algorithms to determine how to pair stylists with clients and to help buyers predict what will be in style months into the future, so they can better manage inventory. For each client, Stitch Fix uses a complex set of algorithms to decode "latent style"—the types of items a client actually likes, regardless of how that client self-labels their style—and then display those preferences on a map that clusters items by style, with "boho" in one corner and "preppy" in another. It also uses algorithms to understand each client's "latent size," since not every medium-size shirt will fit every medium-size client the same way. Over time, these algorithms help to deliver clothes that are likely to best fit the idiosyncrasies of an individual's body—even in extreme cases, which Colson refers to as the "Michael Phelps problem" in reference to the atypical body shape of the world-champion swimmer.

These algorithms supplement the human touch of 3,000 Stitch Fix stylists, who use the data to curate each client's box of clothes. The company has become pretty good at predicting what people want: Nearly 90 percent of its clients are repeat buyers and, anecdotally, the company seems to understand the particulars of sizing and personal style that can make shopping so difficult.

"We're getting really good at the item problem. But it's so much more complex when you start to say which items go together," says Hilary Parker, a data scientist on Stitch Fix's recommendations team who built out the dataset that powers the Shop Your Looks algorithm.

Keep Reading The latest on artificial intelligence , from machine learning to computer vision and more

Like many of Stitch Fix's clients, Parker says she felt confident with "object shopping"—finding individual items that she liked—but struggled to put those pieces together. "I'd buy them and hang them in my closet and admire them, like a sculpture," she says. "It seemed like a good idea to explore that more algorithmically."

First, Parker needed to find data about what kinds of outfits "worked." Stitch Fix has an inventory of millions of items scattered across its six warehouses. Modeling every possible outfit set would be impossible, and even if it were, most of those outfits wouldn't make sense. Instead, Parker enlisted the human stylists.

These stylists already create an "outfit card" around each item that goes out in a client's box. It shows how a singular piece could be styled into a complete outfit, drawing in data about a client's personal preferences. For Parker, it set off a light bulb. "That's data that we hadn't been leveraging," she says.

The trick was to figure out how to scale the one-off outfit cards, which are hand-curated by each stylist, into a database of many outfits that showed the versatility of each item. To generate more examples of how a human might style a particular blouse or pair of jeans, Parker and user-interface engineer Divya Prabhakar built an "outfit picker" tool. The tool asks Stitch Fix's stylists—as well as members of the merchandising and creative teams—to create outfits based on various prompts, like "date night outfit" or "trend-forward outfit." Each prompt starts with a single item and then lets them mix-and-match items from Stitch Fix's inventory.

"It's like a real-life Clueless closet," says Parker.

This growing database of completed outfits created the basis of a machine learning model that predicts which items go together. It works similarly to the way machine learning models can recognize disease from medical imagery or suss out spam text in emails—by picking up on patterns in the data. In Stitch Fix's case, that includes patterns in color combinations, fabric patterns, and silhouette.