Heightened competition in the market for legal recreational weed is pushing prices down and firing up border wars across state lines. Competition is so steep in Washington state that cannabis revenue growth has slowed as Oregon ramps up its adult pot sales.

What's more, in Colorado and Oregon there's more interest from out-of-state investors entering the recreational pot business.

Then there's California, where voters will decide later this year whether to legalize recreational pot.

Yet, for all the interest, the marijuana business remains fraught with risks and capital challenges.

"Cannabis is still illegal on the federal level, so it makes investing very risky," said Jessica Rabe, a research analyst with ConvergEx, a New York-based brokerage that released its outlook for the industry last week. Federal laws make it difficult for marijuana businesses to open bank accounts, and as a result many have gobs of cash sitting around, making them targets for robberies.

"The recreational marijuana business shows a vibrant and growing industry as more states legalize the drug," ConvergEx said. However, it said several states with legalized recreational use have put barriers to out-of-state investors, and "the U.S. Department of Justice frowns upon outside money flowing into the marijuana industry as well."

Read More from CNBC: Why the Election Will Be Big for Pot Business This Year

Let our news meet your inbox. The news and stories that matters, delivered weekday mornings. This site is protected by recaptcha

Overall, state-regulated marijuana sales reached $5.7 billion in 2015, up nearly 24 percent from the prior year, boosted by growth tripling in the recreational market, according to "The State of Legal Marijuana Markets" report from ArcView Group, an Oakland, California-based investment network for cannabis start-ups. It forecasts legal cannabis sales are on track to reach $22.8 billion by 2020, with adult use representing more than half the total market.

Nearly half of the states and the District of Columbia have passed legislation or ballot measures to legalize the possession and distribution of marijuana for medical purposes, and four states — Washington, Oregon, Alaska and Oregon — have legalized adult recreational use. At least nine states have medical cannabis or adult legalization measures in the works, including the closely watched ballot initiative effort in California to legalize recreational use statewide for people 21 and older.

Meantime, a U.S. Senate drug caucus hearing set for Tuesday underscores some of the uncertainties at the federal level still facing the marijuana industry.

Sen. Chuck Grassley, R-Iowa, chairman of the Judiciary Committee and the Caucus on International Narcotics Control, and Sen. Dianne Feinstein, D-Calif., co-chair of the drug caucus, are convening the hearing entitled "Is the Department of Justice Adequately Protecting the Public from the Impact of State Recreational Marijuana Legalization?" Both lawmakers have long been opposed to legalizing recreational use.

The Senate hearing follows the release of a marijuana report last month by the Government Accountability Office that Grassley and Feinstein had requested. The GAO report called out the Department of Justice for not providing more clarity on its marijuana enforcement policies and recommends "DOJ document a plan specifying its process for monitoring the effects of state marijuana legalization."

"They are really barking up the wrong tree if their opinion is that the Justice Department should be going after people who are following state law in states that have decided to go this route," said ArcView CEO Troy Dayton. "The public is behind us, and at the end of the day public officials ultimately answer to the public."

California's recreational use initiative imposes a 15 percent tax on retail sales of marijuana and includes cultivation fees. Recreational legalization backers have raised more than $2 million in campaign funds, including about $1 million from Silicon Valley billionaire Sean Parker. Also, Lt. Gov. Gavin Newsom, a candidate for governor in 2018, has endorsed the measure.

Read More from CNBC: Whoopi Goldberg Launches Pot Products for Women

"For California, you're looking at a $15 billion market potentially, and probably 15 percent of that will go back to the state coffers," said Norton Arbelaez, an entrepreneur and attorney who founded Denver-based RiverRock Cannabis, a chain of marijuana dispensaries. He said passage of recreational marijuana in Colorado created a fast-growing industry with more than 10,000 jobs, and he believes the economic impact in California would be much greater because "California's economy is the size of France."

California voters approved medical cannabis in 1996, and several previous attempts to legalize recreational use in the state have failed. Jason Kinney, a spokesman for the adult weed legalization initiative in California, said the group has "every expectation" the measure will qualify for the November ballot.

The medical market in California, according to ArcView, is expected to remain the largest nationally through 2020, although it projects the medical market portion will shrink from $2.7 billion in 2015 to $2.6 billion by 2020. It cited strict new medical regulations passed in 2015 and the legalization of adult use in 2016, which is expected to reduce the levels of patient participation.

Retail recreational marijuana sales in Colorado and Washington state last year were roughly a combined $1 billion (when adding medical marijuana sales the combined total regulated sales in Washington and Colorado reached $1.7 billion in 2015, with Colorado almost $1 billion by itself). Oregon figures appear to show the first month of adult use in January 2016 produced about $14 million in retail sales, outpacing the first-month sales from legal weed in Colorado and Washington.

Also, there's been downward pricing pressure on legal marijuana in Washington state and Colorado, and analysts expect Oregon to start to see it later this year when more retailers are allowed to sell recreational pot. Furthermore, there are reports of competition across state lines, including marijuana retailers in southern Washington facing increasing competition from Oregon, where pot taxes are lower and set to decline again later this year. ArcView's report referred to "cross-border wars" across state lines and revealed that "immediately following Oregon's market launch, Washington's average daily cannabis revenues began to fall by almost the exactly same amount as they were once growing."

Read More from CNBC: Bernie Sanders Files Bill to Legalize Marijuana

As for Alaska, the state plans to issue recreational licenses later this year. Already, nearly 240 business applications have been received from proposed retailers, cultivators, testers and manufacturers, according to the state.

Alaska has strict marijuana business residency requirements to participate in the industry, and the business must essentially be 100 percent Alaskan-owned (California's proposed adult legalization measure has no such restrictions). As a result, RiverRock's Arbelaez believes the lack of out-of-state investors and expertise will result in "a slower development curve" for Alaska. Still, Alaska is the only state to approve allowing consumption of cannabis-infused edibles on the retailer's premises, setting the stage for Amsterdam-style pot cafes to serve spiked brownies as early as this fall.