Like pretty much every skilled professional in the 21st century, graphic designers now compete not only against their peers but against anybody anywhere who wants to compete with them. Thanks to services like Crowdspring.com, clients large and small can post a design task and set a fee and sit back while far-flung competitors fight for the prize. (Only the winning design earns the fee.) Debates about sites like Crowdspring usually turn on familiar talking points: There are those who say democratization benefits clients by leveraging the market while leveling the playing field for upstarts who lack credentials and connections; then there are those who believe amateurization degrades and undermines a noble profession.

But never mind the morality of the crowd-sourcing systems. What about the results? I was curious, so I signed up for Crowdspring and posted a small design job. And to help me assess the results, I recruited some actual experts: Roman Mars, creator and host of the highly enjoyable design-focused Podcast 99% Invisible; Debbie Millman, president of design at Sterling Brands and chair of the School of Visual Arts Masters in Branding program; and the art department of Slate.

Before we get to the results, a few words about the parameters of my little project. I offered the minimum “reward” (to use the service’s terminology) that Crowdspring allows: $200. Obviously that means I wasn’t expecting to get the same quality that a top-tier firm like Pentagram or a superstar designer like Milton Glaser would presumably deliver. For that matter, offering the minimum fee meant that I didn’t even expect to see the very best work the service could produce.

But that wasn’t the point of this experiment. The point was to see what the minimum gets you. After all, the minimum is both the most threatening price point (as theoretical pressure on professional rates) and the most democratic (putting design within reach of those with limited budgets). My project was conveniently minimal: I wanted a logo for my email newsletter, and I certainly didn’t have the money to hire a pro.

Registering with Crowdspring takes a minute or two, tops. You can lurk around looking at lots of other people’s projects—and in many cases the submissions—for free, but if you want to post a job, you have to pony up: $359 for a “standard” account. (That amount includes the $200 minimum fee for the winning designer; obviously you can pay more if you want to offer a higher fee. If you pay $1,349, you can register as a “Pro” and supposedly get access to “top creatives,” plus more control over who can see your project’s entries, and so on.) I filled out the “creative brief” form, answering Crowdspring’s canned questions (“What are the top 3 things you would like to communicate through your logo?”) as concisely as possible and linked to an issue of the newsletter itself. My entire write-up was 192 words.

After a week, I had 32 submissions, from 22 designers. (All told, there were 39 entries, but some were withdrawn and replaced, or were revised versions of the same design.) That’s nowhere near the “average of 110+” the site bragged about, but my lazy cheapskate approach may have dragged my total down. Either way, what I got was frankly a lot more than I was expecting. And speaking only for myself—my expert judges, as you’ll see, don’t all agree—the quality was better than I would have guessed.

Which isn’t to say I was bowled over. Many of the submissions were easy to eliminate: They looked tossed-off, with icons that didn’t make sense or were obvious clichés. And it was striking how curiously uniform much of the work looked—similar variations on slick, corporate, computer-graphic aesthetics. (One or two evoked Pac-Man, though I assume not intentionally.) There was also a remarkable overrepresentation of the color blue—I’d offered no color specs. CrowdSpring prods customers who post projects to rate entries on a five-star scale and send feedback to the designers. So I narrowed down my own favorites to five (see the accompanying slideshow) and turned to my expert advisers.

Roman Mars, while thoroughly diplomatic, couldn’t get enthusiastic about any of the offerings, zeroing in on a side effect of their uniformity: a general lack of personality. Everything seemed “fine, and clean, and proper, without being right.” One variation, which incorporated some speech balloons with squiggly lines (picking up on my brief’s mention of the newsletter as a means of sparking conversation with readers), struck him as closest to the mark. He described it as “a little messier” than the others, and thus more human.

Debbie Millman was even less impressed: “Nothing remarkable, nothing with charisma or stature or intellect or wit.” She suggested that CrowdSpring’s “four boilerplate questions” (and my somewhat terse answers) were part of the problem: a successful design would require a much more thorough conversation.

My advisers from Slate’s art department were somewhat more upbeat. Holly Allen liked the “visual wit” of a version that converted the M in the newsletter’s name (Consumed) into an envelope icon that also suggested a speech bubble. Natalie Matthews and Slate design director Vivian Selbo liked that one, too, though each had other favorites. Vivian made the case for a type-only version, in which the name fades from one color to another against a reverse background. “It best represents the idea of consuming as an act whereby the thing taken in becomes part of the whole.”

I liked that one, too. But as it happens, this was Roman Mars’ least favorite design: “All I see is ‘Umed’ and think it is a logo for a new university hospital.” That was a good point. But without a clear consensus, I decided to embrace the theory that divisiveness is good: Anything that’s both someone’s favorite and someone’s least favorite must have something to it. My winner (who goes by “designholic” and turns out to be in Brussels) promptly sent me proofs and finals in various digital formats, and my approval prompted Crowdspring to fork over the $200.

I was a more-or-less-satisfied customer—but was I also an immoral one? Millman reminded me that every designer who submits a job to CrowdSpring and doesn’t get picked is working for free: This variation on crowd-sourcing is “exploitative,” and while she was too polite to say so, I’m clearly the exploiter here. Mars, meanwhile, told me that he’d once considered having his 99% Invisible podcast host a contest to design a replacement for San Francisco’s city flag (which he detests), but he was convinced by his design-world contacts that the professional community would flat out oppose this effort, for similar reasons.

With these concerns in mind, I tried to find out a little about the people who had submitted entries to my logo project. The version with the mail icon had come from a “lpavel,” who is apparently in Moldova and who has prevailed in six past CrowdSpring projects; he didn’t seem upset at not winning this one, and offered this link to his online portfolio. Another one of the better entries came from a designer in Bulgaria, whose entries have evidently won 59 CrowdSpring jobs. A third came from a designer who, amusingly, asked me not to reproduce her design in this column—because she’d tweaked it and submitted it to another logo-seeking project for a company with a name that started with a “C.” (It seems to have won her a $400 award.)

It was hard for me not to think about CrowdSpring in the context of the “democratization” of my own profession over the last decade or so. Argue all you want about whether, say, unpaid contributors to the Huffington Post are citizen-journalists storming the barricades of the elite media, mediocre talents being exploited by a profit-hungry business, or both. They’re a fact of life, and I suspect that the same is true of designers in Moldova and Bulgaria taking a shot at getting $200 from some guy with a newsletter. I wouldn’t use CrowdSpring for a more serious design project—but I also wouldn’t have shelled out serious money for this one. As Roman Mars summarized it during our discussion: “It looks like you got what you paid for, which should be reassuring to everyone.”