HVMN, a San Francisco start-up backed by former Yahoo CEO Marissa Mayer and Andreessen Horowitz, advertises its $40-a-bottle supplements as "biohacking" compounds that will help people achieve "optimal human performance." But CNBC has learned that the first clinical trial study commissioned by HVMN (pronounced "human") found that one of its best-selling supplements was less effective in many ways than a cup of coffee. After the disappointing results in May, sources said the company hoped to delay publication of the study and asked researchers to change the name of the product to distance it from the analysis.

Big promises

HVMN sells nootropics, otherwise known as "smart pills," a growing favorite among Silicon Valley's elite, who are looking for ways to function at super-human levels. The company raised about $3 million, and investors include Zynga co-founder Mark Pincus and Kabam CEO Kevin Chou. The company says its supplements such as chewable caffeine pills help the human system become "quantified, optimized, and upgraded." At one point, CEO Geoffrey Woo went as far as to describe HVMN's products as unlocking "next-level thinking" that will be key to humanity's evolution. "In a way, it's almost arming humanity against artificial intelligence and robots," Woo told Bloomberg. All the marketing seems to be working. HVMN is selling some of these supplements in the tens of thousands of units per year, reaching monthly subscription revenue of $3 million to $5 million in 2016, according to a source familiar with the company's financials. HVMN said it generates sales in the "multimillions." But many of its claims lack scientific evidence to support them. Little is known about how the human body responds to the cocktail of ingredients, both natural and synthetic, that are found in most nootropic blends. So in 2016, HVMN -- known as Nootrobox at the time -- set out to prove itself by testing one of its supplements, a "cognitive enhancement" called SPRINT, against caffeine in a landmark clinical trial. It commissioned a study in collaboration with Maastricht University in the Netherlands. HVMN was hoping to support its claim that SPRINT can help people "conquer a big project, a long day at work or any other mentally demanding task." The results? In most areas, the supplement tested was less effective than sipping a cup of coffee. "As we expected, the caffeine had some positive effects, but the SPRINT formulation they gave us was not really effective," said Arjan Blokland, head of the department of neuropsychology and psychopharmacology at Maastricht University, in an interview with CNBC.

SPRINT claims to "get the job done"

Plain caffeine was better in almost all cases

CNBC has reviewed the original results of the unpublished study. The published version is expected to run online in the coming weeks. For the research, a small group from Blokland's department at Maastricht assessed memory performance, attention and sensorimotor speed -- meaning sensory and motor rather than cognitive activity. It also measured other things like alertness, heart rate and blood pressure. At the 30 and 90-minute marks, they asked the young and healthy participants to take a verbal learning test to analyze their performance. The study was randomized, double-blind and placebo-controlled, which is generally considered the gold standard in reducing bias. Their conclusion: "In healthy young students, caffeine improves memory performance and sensorimotor speed, whereas SPRINT does not affect the cognitive performance at the dose tested." Specifically, the study found that caffeine was more effective for delayed recall performance, working memory and the speed of responding. In fact, the participants remembered on average 2.4 words fewer on a memory recall test when they had been given a capsule of the tested product versus the caffeine alone.

The capsule was only more effective than caffeine on subjective alertness at the 30-minute mark. It also did not raise the subject's blood pressure as much as caffeine, which is a common side-effect that can create headaches. Woo told CNBC he was "excited" by that finding. "We stand behind the research," Woo said. "We tried to make a study that would show effects. In some cases, it did show positive effects. On some measures, they were negative effects."

'Not an end result we had hoped here'

When Maastricht shared the results in an email to HVMN's founders, Woo wrote in an email to Blokland, "Obviously not an end result that we had hoped here." Woo expressed concerns to the researchers that the study had not controlled against factors like mood. Separately, Woo told CNBC in an email that the result was inconsistent with prior studies that showed caffeine had more effects than a placebo, which made him doubt the validity. "In order to be confident of the sensitivity of the study, we would expect to detect a difference between the 'positive control' of caffeine vs. placebo, consistent with the many previous investigations involving caffeine. Unfortunately this was not the case. This strongly suggests that there may have been issues with the cognitive testing methodology or sensitivity of the study generally, and that the cognitive results ought to be taken as inconclusive." After the disappointing results, Woo emailed the researchers requesting that the product tested not be labeled SPRINT. (Update: HVMN wrote a blog post in response to our story. You can read it here.) But the company certainly expected to be testing SPRINT. It had originally registered the clinical trial in a U.S. government database as "The Effects of SPRINT, a Combination of Natural Ingredients, on Cognition in Healthy Young Volunteer." It also described the formulation in an October 2016 Medium post as SPRINT MK.111, adding that it's an "upgraded composition based on our latest research and data." An independent lab study seen by CNBC suggests the tested compound had slightly less L-Tyrosine than SPRINT, which Woo confirmed. Woo also said the tested compound had more Vitamin B12, but the lab test did not show this difference. Two sources told CNBC that the product tested would have been the next evolution of the SPRINT product. "I think they expected it would perform even better than what they were selling," said Blokland. Woo disputed this, saying that the company was testing "a couple different variants" of SPRINT, and while this one is "very similar" to the product now sold as the third iteration, it is not identical. Regardless, the paper published in January will not refer to "SPRINT." The compound will instead be described as "CAF+" -- short for caffeine-plus.

Many scientists are skeptical