When subscription e-commerce tool Subbly made it to the Product Hunt homepage, it changed everything. Its sign-ups nearly quadrupled. Its paying customer base grew five times. Investors started calling. “It really motivated us,” founder Stefan Pretty told Re/code. “We realized we were onto something.”

It’s a well known startup fable among Silicon Valley insiders, with Product Hunt playing the role of kingmaker, but for those who may be unfamiliar, the two-year-old website has become a surprisingly powerful force in the industry, anointing startups with near royal status. Compelling apps and hardware are listed on the site for others to discover, and, as on Reddit, readers can vote up their favorite submissions.

What results is a daily list of tech products, ranked first to last by how much people like them. Investors and reporters surf it daily for the latest stories and potential investments, making Product Hunt the Valley’s daily scorecard. And for the little-known startups that make the homepage, it can mean thousands of dollars in sales, a multiplied user base, national media coverage or urgent queries from venture capitalists.

But there’s one big hole in the Product Hunt story: For a site designed to empower its users, it’s not exactly clear how the service works.

For every Subbly, there are people who’ve spent weeks or months preparing a product, only to post it to Product Hunt and never see it appear on the homepage. The application process is opaque and confusing, leading to threads of complaints online.

“Is Product Hunt rigged?” asked a Quora user.

“It seems like only the ‘elite’ members get a say,” someone commented on Hacker News.

“The front page of Product Hunt is curated by all the same people upvoting each other’s shit which most of the time entails pandering e-books or how to make GIFs of cats,” a Reddit user agreed.

It’s ironic, given the fact that Product Hunt’s goal from the beginning was to “democratize attention” for startups. And it’s turning out to be a tricky mission to fulfill without ruining the site’s tight-knit community, which, unsurprisingly, is composed largely of men.

By invitation only

Product Hunt is the brainchild of 28-year-old Ryan Hoover, an Oregon native who had previously worked at a gaming company. “I would love to open it up to everyone, but we just can’t,” he told me last year, citing the potential for spam and self-promotional posts. “It would turn into probably the worst site on the Internet.”

Hoover has granted only 1,800 people the power to post directly to the homepage. It’s a small group that was assembled by invitation when Product Hunt launched in 2013. These so-called administrators are Hoover’s friends and professional contacts. Half of the group are “makers” — people who built a product that was featured on the homepage in the early days of the service.

More specifically, the 1,800 include venture capital and seed investors, sought-after entrepreneurs like Marc Andreessen, developers and tech journalists — including me. I wrote about Product Hunt when it launched, which is how I made the list. But I’m not much of a product hound, so I’ve only submitted and up-voted a couple of items (blasphemy, I know). The list also includes celebrities, such as Snoop Dogg, who recently launched an album on the site, and Nas.

For the moment, no one can join this elite group — the list is essentially locked. But on a few rare occasions, Hoover has let someone in. When B.J. Novak, the actor and producer from the TV show “The Office,” joined the service, Hoover granted him homepage posting power.

Anyone can vote products up but it’s this inner circle who decides what people will vote on. As a result, the active administrators, whose profiles appear next to the products they submit, regularly receive emails and tweets from companies hoping to get featured.

There is yet another subset of users, roughly 8,000 people, who can post to Product Hunt’s “upcoming feed,” a separate index of products found within a hidden tab on the site that some have likened to a “black hole” for its lack of visibility. They’ve been granted access to that feed by other active Product Hunt members, whom the company gives invitations to dole out. This group is a junior varsity of sorts, as Product Hunt staff will occasionally pick submissions to feature. About 60 percent of what appears on the homepage started out in this “upcoming feed.”

Altogether, more than 23,000 products have been submitted to the site, with 243,000 registered users casting votes. That may not seem like a lot, but Product Hunt directs 3.5 million unique referrals per month to other sites. That’s how Hoover tracks Product Hunt’s impact since its whole purpose is to enable product discovery.

Let’s not upset the users

But like many of the startups it features, Product Hunt is also a business in search of a business model. Hoover has raised $6.1 million from venture capital firm Andreessen Horowitz, along with Slow Ventures, SV Angel and individual investors like Alexis Ohanian, founder of Reddit. It has 17 full-time employees — designers, coders, company administrators, HR staff and site moderators.

Despite the site’s outsize influence, Hoover’s company is still sorting out how it will make money. It currently has no revenue, but Hoover is exploring several options: Advertising, job recruitment listings and affiliate revenue from the apps people download after having discovered them through Product Hunt. The team isn’t planning on rolling out any of these any time soon, choosing instead to focus on growth.

Part of the hesitation is because Hoover is wary of the pitfalls that come from the need to generate revenue. He has spent time with Digg founder Kevin Rose, an investor in Product Hunt by way of the firm Google Ventures, to better understand how overt advertising contributed to Digg’s decline. “The hardest part is finding that balance and not hurting what’s working,” Hoover said. “The most difficult thing will be ensuring that whatever we do is okay with the community.”

The company aims to grow by launching other product categories. It unveiled a gaming tab last week, and more are likely to come. Hoover thinks new sections will help with the democratization of Product Hunt by making the service appealing for all types of people.

“Silicon Valley is such a small world — just because we don’t think something is cool doesn’t mean the majority of the world won’t find it valuable,” Hoover told me when he raised funding last year. “There’s a lot of crazy people with different interests, crazy people that love cameras and gadgets and scrapbooking, and people who live and breathe hip hop.”

An outsider to the scene

But for the moment, the community that drives the site is one built largely in Hoover’s image. As he’s chosen the site’s initial administrators, most of whom have remained the core homepage submitters, the service is arguably a bigger reflection of his own personality and preferences.

Hoover, who is from Eugene, Ore., and studied business at the University of Oregon, comes off earnest and positive, which he attributes to his natural disposition. Product Hunt itself has a “no hating” policy, and those who insult others in the comments section are gently reprimanded. The people Hoover admires in tech, people like Path’s Dave Morin (also a Product Hunt investor), are active participants.

Before starting Product Hunt, Hoover was with gaming company PlayHaven, which builds tools for game developers, where he worked for more than three years as a product director, adding to his Silicon Valley network.

Five years after he moved to San Francisco, Hoover has become one of the startup scene’s most important arbiters. Even so, he’s hardly the kind of ideologue in the way that previous versions of the blogger-kingmaker, most notably Michael Arrington of TechCrunch, once wielded influence. At least not directly. Product Hunt is a user-driven content site, ostensibly relegating Hoover to one of its many functions.

It’s also possible he just prefers it that way, because in some ways Hoover still acts like an outsider to the scene. Last year, at the company’s second ever Happy Hour, which attracted hundreds of entrepreneurs (compared with the next one, which attracted thousands), he left after only a few drinks. When I asked him why he left so soon, he became defensive: “I stayed for hours! I have to wake up early tomorrow.” In recent months, he’s started avoiding public events altogether, sending Erik Torenberg, a member of his founding team, in his stead.

He tries to pour all his time into developing the service. “I’m very cognizant that things can go south,” Hoover said. “The worst thing you can do is something that pisses everyone off. That’s why I’m on that hamster wheel always running.”

Still a black box

Not everyone is impressed with his efforts. “The curation process isn’t transparent — why is this product selected while another one isn’t?” said Max Woolf, a Bay Area based software engineer who has written about the issue on Hacker News, an online forum. “A lot of the products being submitted pander to the demographic of the Millennial in Silicon Valley.”

There does appear to be a self-perpetuating aspect to the site’s selections. When someone posts a product to the homepage, it sends a notification to his or her followers (Product Hunt members can be followed as on Twitter), making product submission a performative social act. Product Hunt user Jeff Morris said, “When people ask me to post [their app], I’m thinking, ‘Do I want my friends and network to see that I’m endorsing this product?'”

“It’s just a judgment call,” Hoover said when asked about his own process for picking products. “It’s based on being on Product Hunt and knowing the community.” He pays attention to how many votes products receive and then prioritizes similar ones when they surface later. Chrome extensions, Mac apps and Slack integrations are all big hits.

He gives priority to bigger-name companies, like Facebook, or well known techies, like Rose, because he knows Product Hunt’s audience cares about them. That’s in part where charges of elitism come from.

Hoover hopes to automate the process at some point so it really is run by the community, but he wants to spend time building the right solution. “We’re playing the long game and constantly trying to balance what we need to do,” Hoover said. “What’s more important right now is to grow and make sure our community is stable and healthy.”

It’s a man’s, man’s, man’s world…for now

Product Hunt doesn’t keep statistics on the gender or ethnic breakdown of its 1,800 homepage submitters. Hoover could only offer one figure — 26 percent of them live outside the United States. He pointed that out because Product Hunt employees occasionally bump items from the upcoming feed, 40 percent of the items appearing on the homepage were originally submitted from people outside the U.S.

The gender breakdown appears to be far from egalitarian. A quick scan of the service on one weekday showed only three women represented out of 25 submissions. Over the weekend it was two women out of 17 submissions.

It’s not that Hoover doesn’t care about female representation. Multiple times, he has appealed to me personally to use my posting privileges to submit products. I’ve done so only on a few occasions. “There’s no reason to force women to post,” he said when I wondered why he stopped asking me. “If you don’t like the content, why should I beg?”

He hopes that changes to the site, like the recent introduction of more visual elements and an eventual expansion into new categories, might attract more women to the service.

Not for everyone

Those who have seen success on Product Hunt are its rabid defenders. “It’s giving us real credibility,” Subbly’s founder Pretty said. “We’re raising funding, and being able to put Product Hunt on our pitch deck is serving us quite well in terms of getting that instant reputation.”

Others haven’t been so lucky. Alex Flom, founder of Kilometer.io, which offers tools to help startup founders, never made it to the homepage despite active lobbying. “I wish that everyone could submit products and let the visitors decide what’s good and what’s not good enough to make it to the front page,” he said.

“There are probably lots of other great products out there that people just can’t submit,” he added.

But as much as Hoover wants to open up the floodgates, he’s fundamentally stuck. He feels that if he were to give submission privileges to anyone and everyone, the homepage might become overrun with marketing.

“Ultimately there has to be some connection and interest and bond,” Hoover said. “Product Hunt should never appeal to everyone in the world.”