Last year, West Virginia did something no other U.S. state had done in a federal election before: It allowed overseas voters the option to cast absentee ballots for the midterm election via a blockchain-enabled mobile app. According to Voatz, the company West Virginia worked with, 144 individuals from 31 countries successfully submitted ballots via the app for the November election. Before that, there was a smaller pilot of the system in two West Virginia counties that May.

West Virginia billed the experiment as a success and says it plans to use the technology again in 2020. Voatz has already made deals with other local governments in the U.S., most recently for Denver’s May municipal election.

But how secure and accurate was the 2018 vote? It’s impossible to tell because the state and the company aren’t sharing the basic information experts say is necessary to properly evaluate whether the blockchain voting pilot was actually a resounding success. With 2020 looming, that’s troubling, given what we now know about the extent of Russian incursions into our election systems in 2016.

State officials in West Virginia said the goal of rolling out the mobile voting option was to make voting easier for troops living abroad. But West Virginians overseas didn’t have to be in the military to take advantage of the process. All citizens had to do was register, download the app, go through a few verification steps such as uploading a photo ID and taking a video selfie, and make and submit their ballot selections on the screen. And all of it was said to be secure. With the blockchain technology it used, the firm insisted, the votes would be near-impossible to hack. (Blockchain is a digital public ledger that records information. It can be shared and used by a large, decentralized network, so it is theoretically more resistant to tampering.)

But numerous election technology experts sounded the alarm over what they said was the enormous potential for glitches and security risks on people’s mobile devices, the networks that hosted them, and the servers that held their information. Amid the words and phrases they used to describe West Virginia’s experiment: “horrible,” “horrific,” “completely nuts,” “high-flying blockchain promises,” “the Theranos of voting,” and “no.” Some pointed to the lack of transparency around the app, others to the inherent weaknesses of conducting an election over the internet at all.

Our closest look at the details of the voting experiments came in an eight-page white paper published in February, but it was short on details. In the report, Voatz said that it had retained four independent security experts to audit its system. Though the white paper included “fun facts for election geeks,” it failed to name any of these auditors. Also unlisted: the scope of the tests conducted, what exactly the auditors had access to, how long they had to perform the tests, what vulnerabilities were discovered, the severity of those vulnerabilities, and whether or not they were fixed.

When asked about why a redacted audit or report from the auditors wasn’t released, Voatz co-founder and CEO Nimit Sawhney, who co-authored the white paper, gave multiple reasons. First, he pointed to a nondisclosure agreement with the auditors. Then, he stated that there was no way to share any more information about the audit without revealing proprietary information about the system. But one would expect a redacted report or even an abstract of the report for transparency’s sake, given the stakes of introducing a new system to our already rickety voting process. Depending on the secrecy of its system architecture is such a poor security mechanism that researchers have even coined a term for it: security by obscurity.

To its credit, Tusk Philanthropies, the organization that funded most of Voatz’s mobile voting pilots, contracted ShiftState Security to conduct its own, separate audit and to review the other audits and penetration tests (where auditors look for security vulnerabilities that attackers could exploit). ShiftState Chief Security Officer Andre McGregor told me that the firm deployed “a couple of consultants,” including himself, over a month to conduct a full security review, interview Voatz employees, and see whether penetration tests conducted by another firm were in line with the results one would expect from a pen test of that type of software stack. McGregor said Voatz “did very well” in the audit. But he declined to answer several questions because of a nondisclosure agreement he had signed with Voatz. After Voatz Senior Vice President Larry Moore told me that the company would release McGregor from his NDA, McGregor stated that any interview questions would need to be sent via email, but he did not respond to those.

Voatz’s lack of transparency makes it hard to check if votes were tabulated accurately. The most important part of conducting a post-vote tabulation audit—a way to achieve confidence that reported election results are correct—requires taking a random sample of ballots that reflect the true intent of the voter. This is where having a paper trail from voting can come in handy. With machines that generate paper ballots or traditional paper-based voting, you can hold the piece of paper and verify it before putting it into a scanning tabulator.

Voatz’s website states that “a paper ballot is generated on election night” and is tallied “using the standard counting process at each participating county.” What that means is the voter’s vote is sent to the county clerk staff as a PDF, and the county clerk staff prints it out and puts it into the scanning tabulator. Those paper ballots generated from the PDF might still be useful to audit the tabulator itself, but not the vote casting process, since the voter never had a chance to review the paper directly. This requires the voter to trust that the government will do all this correctly behind the scenes.

“There isn’t anything like that to check when you’re using internet voting or mobile phone voting, so there’s no way to go back and ensure that the tabulation was correct and that it matches something the voter intends,” said Audrey Malagon, mathematical adviser for Verified Voting, a nonpartisan nonprofit that advocates for regulation and legislation promoting accuracy, transparency, and verifiability of elections.

Voatz’s website also says that a postelection audit can compare “the paper ballots with the anonymized voter-verified digital receipts generated at the time of vote submission.” But a carbon copy of a mobile or online vote is not a paper ballot, and comparing that receipt with the results is not a tabulation audit. “In order to have an authentic tabulation audit, you have to be able to audit something that the voter has been able to verify, and so any time you’re introducing these extra steps and extra processes, you have the extra potential for error,” Malagon said.

Sawhney says that each voter receives a verifiable receipt of their vote and has the ability to check whether it represents their intent, and an anonymous carbon copy of the email is sent to the jurisdiction. The tabulation audits have been conducted based on the copies sent to the jurisdiction. But what if the signature scheme is broken or an email wasn’t properly sent to both recipients? “The only thing a voter can actually verify is something that the voter sees. If the voter is in one place looking at a voter-verifiable receipt and someone else is in another place looking at a printed ballot that supposedly matches that receipt, how do we know that it actually does?” asked Mark Lindeman, Verified Voting’s senior science and technology policy officer.

Voatz’s use of blockchain doesn’t solve this problem. Advocates of blockchain voting point out that the blockchain is resistant to tampering, which they say can protect the process. But anyone who has voted in this way might want to check their vote in the blockchain to make sure it’s actually there and says what they intended. Although Voatz is hoping to change this in the future, its users currently have no way of doing so. And generating paper ballots that the voters are unable to see or verify is not enough.

Even if Voatz does manage to build a viewer allowing voters to verify their votes within the blockchain, it still doesn’t solve the problem of tabulation audits, where using random samples is part of the point. And it’s not possible to ascertain whether carbon copies of voting receipts reflect voters’ intent the way paper ballots can.

Finding a technical solution that will allow voting systems to show that what’s being recorded, counted, and stored in the blockchain reflects the voter’s intent without compromising the voter’s secrecy isn’t easy. That’s why many experts think that blockchain isn’t an appropriate tool for voting, at least not yet.

Voatz has promised additional white papers, and that future audits will be conducted by third-party firms and will be in the public domain before 2020. If that happens, it would be a step in the right direction. An audit of the vote in Denver was conducted by the National Cybersecurity Center, a nonprofit organization established in a bill signed by former Colorado Gov. John Hickenlooper, who is the founding director and has a nonvoting seat on the board. The tabulation audit was, again, conducted based on receipts. The audit itself was, at least, more transparent, with an audit process demo and video posted publicly, along with a Facebook Live video by the Denver Elections Division.

More audits by the U.S. Department of Homeland Security are planned too, though it is unlikely that the results will be shared publicly in a meaningful way. Donald Kersey, general counsel and former elections director/deputy legal counsel for the West Virginia Secretary of State’s Office, said that penetration testing will be done by the Department of Homeland Security in the summer and fall, and that tests will include on-site internal intrusion attempts in the fall. This will generate an audit report, but not one that’s public, though Kersey plans to summarize it for voters. When asked how voters can determine whether he is cherry-picking from the report, Kersey said, “If there’s something insecure, we want to know about it, and if it can’t be mitigated, we’re not going to use it.”

“Nobody’s interested in putting out something that is not safe and not up to a certain standard,” said Sawhney. “We believe it is pretty robust and does what it claims to do.”

But “pretty robust” is not good enough when it comes to voting securely. And without seeing a proper tabulation audit or other details about how it works, we simply can’t evaluate it.

Future Tense is a partnership of Slate, New America, and Arizona State University that examines emerging technologies, public policy, and society.