Washington (CNN) President Donald Trump's election integrity commission won't have access to all of the information it would like because of state laws that map out what is and what is not publicly available -- triggering a national conversation on the privacy of voters' information.

At the heart of the issue is a letter sent last week by Kansas Secretary of State Kris Kobach in his capacity as vice chairman of the Presidential Advisory Commission on Election Integrity to all 50 states and the District of Columbia. In that letter, Kobach asked for all "publicly available" data, but the long list of pieces of information sought, including the last the four digits of Social Security numbers, included several elements that very few states, if any, say they can legally comply with.

The letter said the commission is seeking to collect feedback from the states that can be used in its work studying the US election system's integrity. But a request for certain pieces of voter information -- especially the last four digits of Social Security numbers, military status, possible felony convictions and dates of birth -- raised the concern of states, privacy experts and cybersecurity professionals.

The federal government and Kobach say the letter clearly acknowledges that states are governed by their own laws in what they can provide, reiterating that fact in a court filing Wednesday against a lawsuit from a privacy group seeking to block the request.

In a statement on Wednesday afternoon, Kobach said 20 states will provide public information and another 16 are reviewing their options, while 14 states and DC have outright refused to comply.

"Despite media distortions and obstruction by a handful of state politicians, this bipartisan commission on election integrity will continue its work to gather the facts through public records requests to ensure the integrity of each American's vote because the public has a right to know," Kobach said.

But states and privacy experts remain concerned by the request in general, and some say the context of how the data is intended to be used matters just as much as the patchwork of privacy laws that could affect what states are able to disclose.

According to a CNN inquiry to each state earlier this week, at least 45 states and DC said they would be unable to provide at least some of the information or outright refused to comply, including Kobach's own office, which can't provide all of the data under state law. A small number of states did respond positively, and some had yet to receive the letter or were still reviewing it.

Is my voting information public?

The short answer is yes, some of it.

Generally, states consider voters' names and addresses public information, as well as whether they have voted in past elections and what party they have registered with.

How a person actually voted in an election is never recorded, though which ballot they requested in a primary election may be.

But from there, it can get complicated. Every state has different laws about how that information can be requested and used. Some states post the information in an online database, whereas others have tight restrictions on who can request it, how and for what purposes.

Some states do consider information like some birthday information and identifiers like the last four digits of Social Security numbers public, but many consider it confidential or sensitive. Information like military membership and felony records is almost never considered part of a voter file.

In addition to state records laws, there are also a myriad of state data breach laws and specific federal privacy laws potentially at play, including privacy laws over drivers' information that could be applicable if voter rolls are mixed with DMV records.

"Voters in most states should feel very confident that at most their name and address is part of their public voter file, and they should feel good that states are being careful custodians of their data to make sure that sensitive information doesn't get out," said David J. Becker, executive director of the Center for Election Innovation and Research and a former director at The Pew Charitable Trusts who ran their Electronic Registration Information Center. "The reason is because political campaigns, the candidates, want to be able to reach out to voters and talk to them, so it's for no nefarious reasons they're available, it's for good reason."

Some of this information is also already aggregated in private, by political parties and commercial vendors who advertise the ability to look people up for a nominal fee, though that information doesn't necessarily come just from voting rolls.

Some states also do provide exemptions for people with a safety concern about their information being public, such as judges, law enforcement officials and domestic violence survivors.

Who can access it?

It depends.

"Public" here has different meanings. Even if a state considers data public record, it can't always be used by anyone for any purpose. Commercial purposes are not allowed, and some states only allow verified political organizations to request the information, which wouldn't include the federal government.

Many states charge a fee for the data, as well, ranging from just a few dollars to tens of thousands.

According to University of Florida professor Michael McDonald, who studies voter rolls and elections, collecting voter information for all the states could cost more than $100,000 if states don't waive fees. His research in 2015 concluded it would cost $126,482 for an average citizen to buy data from every state that allows it, $135,132 for a political nonprofit and $136,671 for parties, candidates or political committees.

In some cases, McDonald told CNN, the cost is designed to prevent "casual use" of the data, but in other states, incumbents are given the data free while outside candidates have to pay, using the cost for a politically protective function.

One wrinkle for the commission is that because of federal open record laws, anything states turn over will become publicly accessible. It was not in the letter seeking the information, but in a court filing on Wednesday, Kobach said the plan is for the commission to "de-identify" any data that is made public, though he did not provide specifics.

That disclosure aspect could complicate what's allowed under state laws.

"The concern that some of these states could have is every state has a prohibition on the voter file being used for commercial purposes, and if the data could be subject to a FOIA request, then the states lose control over who has access to that data," McDonald said.

So if it's public, why did this request cause so much drama?

There are layers of concerns that states and experts have.

Strong responses have spanned both parties. They include Republican Mississippi Secretary of State Delbert Hosemann who said, "My reply would be: They can go jump in the Gulf of Mexico, and Mississippi is a great state to launch from," and Democratic Virginia Gov. Terry McAuliffe saying, "At best this commission was set up as a pretext to validate Donald Trump's alternative election facts, and at worst is a tool to commit large-scale voter suppression."

A small number of states have responded positively, namely Missouri, Colorado and Tennessee.

Some object to the federal government possessing such a robust database of all Americans' voting info and potentially sharing that broadly across the government, especially when it is considered a strength of the American election system that it is dispersed and managed independently by states.

Others object to the very notion that voter fraud is a massive problem. Trump has made repeated claims about millions of illegal votes, yet study after study has shown almost no voter fraud in the US and there were no indicators of any irregularities in the last election.

Another subset are concerned about the data privacy and security consequences of having all that sensitive information in one place -- a very attractive target for hackers of all kinds -- without more information about how it will be protected. Critics noted that the letter's suggestion of submitting the information by email was a red flag, though there was also a federal computer system offered as a different option.

And a further concern is the motivations of the request for information and what the data will be used for, which are not clearly spelled out in the letter. Kobach has been a target of voting rights activists for years, who have accused him of aggressive tactics purportedly to combat mass voter fraud, which studies have found is statistically nonexistent in US elections. The letter, though, does include goals like preventing voter intimidation and disenfranchisement, even if those were not mentioned in the executive order creating the commission to examine voting integrity

Kobach pioneered the growth of a multi-state program, called Crosscheck, that has been criticized by voting rights advocates for using incomplete information that generates thousands of false positives,or false matches of different people as the same voter, perpetuating a myth of widespread fraud and resulting in unjustified purging of voters from rolls.

ERIC, a similar program from Pew, spent years working with privacy, technical and election experts to develop an encrypted database that checks more than name and address to identify potential voter overlap among states, according to Becker and its current director, former Washington elections director Shane Hamlin. It took that long to convince states that sharing data through the program would be technically safe and responsibly handled.

Joseph Lorenzo Hall, chief technologist at the pro-privacy Center for Democracy and Technology and an expert in election integrity, said a responsible scientific proposal would first lay out what it hopes to do with the data and find within it, thus determining the minimal essential data to accomplish that goal. Hall is an adviser to ERIC.

"It looks like this commission just skipped that whole important step," Becker said. "Even if some of this information is public, what are you going to do with it that's useful that merits putting a national database together? States are asking the right questions: They're saying if you want this stuff and you're going to put it together in one place where anyone can get it all at once, you're going to need to demonstrate a real need to us to get it."

"I think there's valid concerns about the security of the data once it's provided and the method of transferring this data, and there are secretaries that are concerned about what the data is going to be used for, such as a national matching program which is much more complex than people realize," Hamlin said. "It's easy to generate thousands of false positives if you don't have good data and good matching processes. And the risk is with thousands of false positives with this data, which won't even be complete in all cases, (and) you can't unring that bell."

CORRECTION: A previous version of this story misstated Shane Hamlin's name.