If New Jersey follows through on plans to legalize recreational marijuana, analysts believe the Garden State could grow into a $1 billion market.

Democrat Phil Murphy, who was elected Tuesday as New Jersey's next governor, vowed on the campaign to legalize marijuana and use a potential $300 million tax revenue to fund education programs and worker pensions.

Legislators there are taking him at his word. State Senate President Stephen Sweeney said he wants call a vote on an a draft bill that was written in May, and get it approved within 100 days of Murphy's inauguration on January 16. Murphy did not return a request for comment from CNNMoney.

Getting to $300 million in tax revenue could take some time.

"The uncertainty at this point is how long it will take from the legislation being signed to stores selling product," said Jon Whiten, spokesman for the New Jersey Policy Perspective, a think tank that's making recommendations on the recreational marijuana infrastructure. "Some industry folks are suggesting at least one year to write the regulations. So, bottom line, $300 million a year in [tax] revenue isn't likely until 2020 at the earliest, but more like 2021."

Getting to $1.2 billion in annual sales by 2021 could be optimistic. New Frontier Data, a company that analyzes the cannabis industry, is more cautious in its sales forecast. It predicts sales wouldn't top $1 billion until 2023 and could fall short of reaching $1.2 billion by 2025.

But Kate Bell, the New Jersey specialist for the Marijuana Policy Project, thinks these analysts could be underestimating the market because their data is based on a federal government poll that showed only 4% of New Jersey residents claim to use marijuana.

She said that's below the national average, which is closer tor 10%, according to a 2015 study from the National Institutes of Health, and it doesn't take into account cannabis-using visitors from neighboring New York City and Philadelphia, who likely will inflate sales.

Implementing tax on legal marijuana can be tricky because there is no tax on the black market.

A study from New Jersey United for Marijuana Reform, a pro-legalization group, and the New Jersey Policy Perspective recommends a phased tax, starting with 5% sales tax the first year, followed by 15% the second year, and 25% in the third.

In other states like Colorado, high sales taxes scared away many customers who continue to make illegal purchases.

"Lawmakers should set a tax rate that promotes public safety and keeps participation in the illegal market as low as possible after legalization," said the study, which doesn't examine other types of taxes, just sales tax.

Related: How much do pot jobs pay?

Even if legislators set a competitive tax rate, building the infrastructure for New Jersey's legal marijuana industry won't be easy, and it won't happen quickly.

Medical marijuana is already legal in New Jersey, but there are only six dispensaries serving a state of nine million people.

"Six businesses is not enough for the entire state of New Jersey, obviously," said Bell.

Eight states and Washington, D.C., have legalized recreational marijuana, and five of those states have already started up retail markets. In that process, medical marijuana dispensaries typically morph into businesses that sell to recreational customers as well, but at higher tax rates, and therefore higher prices.

New retail businesses may open up as well, if they can negotiate the complicated regulatory process.

Related: California pot tax could be 45%

If the state moves quickly to legalize recreational marijuana and if they allow the medical dispensaries an "early start" in selling adult-use pot, then "you could see sales as early as late 2018," said Bell. "But it would take longer to set up new stores."

The legal industry would have to be built from the ground up, from the planting of the cannabis seeds to the growth of the plants to the processing into buds or other products like oils and edibles, the transportation of product from the farm to the store and then retail sale to the consumer.

In this way, New Jersey more closely resembles Oregon, which had a limited medical marijuana industry with a scattering of dispensaries prior to legalization of recreational marijuana in 2012.

"Oregon took two years, and that seems like a reasonable time line," said Bell. "Absolute best case scenario, you're looking at late 2019 [for statewide marijuana sales.]"