Nathan and Patty Benjamin, owners of Charlton Orchards, attended Wednesday night’s Charlton Planning Board meeting. The family’s orchard could be the future home of a 1 million-square-foot marijuana growing facility. Matthew Healey for The Boston Globe

CHARLTON — A controversial proposal to build one of the largest marijuana cultivation facilities in the country here was dealt a setback Wednesday night when the town’s planning board voted to deny a critical approval needed for the $100 million project to go forward.

The board’s 4-0 vote against a site plan submitted by cannabis company Valley Green Grow followed a hearing at which residents loudly jeered town officials and objected vociferously to last-minute changes to the project.

Planning officials -- seemingly defying the advice of their own lawyer, who cautioned such a move might invite legal repercussions for the town -- concluded that the massive proposed facility was too industrial in nature for the property, which is currently a farm.

The company immediately vowed to appeal the decision, likely to the town’s zoning board of appeals.

“They’re not interpreting their own bylaw, they’re just going with their gut, with their instinct,” Michael Rosen, Valley Green Grow’s attorney, said moments after the vote. “By denying the application, they now run that risk it will get approved anyway on appeal, in which case they won’t have the opportunity to impose reasonable conditions. From our perspective, we would much rather have worked it out with them.”

Valley Green Grow hopes to construct a 1-million-square-foot indoor marijuana-growing facility on the site of what is now Charlton Orchards Farm & Winery. The farm’s owner, Nathan Benjamin Jr., has said he wants to sell the property to the company after a 2015 fire destroyed the orchard’s barn and wine-making facility.

Gerard Russell, a newspaper editor and neighbor of the farm who has helped lead opposition to the marijuana project, hailed the decision, saying it was a rebuke to a hurried initial approval last year by Charlton’s Board of Selectmen. He also vowed to continue fighting any appeals by the firm.

“It’s been a dirty process,” Russell said. “Thankfully the planning board tonight did the right thing. It’s an industrial process that’s not allowed by zoning.”

Wednesday night’s meeting of the Charlton Planning Board was at Charlton Middle School. Matthew Healey for The Boston Globe

Russell and other neighbors have tried to block the project from every angle, arguing it would change the character of this Central Massachusetts town and reduce the value of their homes. They’ve swarmed public hearings on the project to air their complaints — ranging from concerns about possible odor from the facility, environmental impacts, and traffic to nitty-gritty gripes over drainage easements.

In July, a group of residents sued Charlton’s Board of Selectmen for allegedly violating the state’s Open Meeting Law by posting an agenda they say was too vague before approving a host community agreement with Valley Green Grow.

The company has also sued the town, seeking to overturn a general bylaw banning marijuana facilities in Charlton that was approved by a majority of Town Meeting voters last summer. The company argues the ban should have been enacted as a zoning by-law, which requires a two-thirds majority vote to be adopted. A hearing in that case is scheduled to take place in state Land Court on Friday.

The proponents of the project said they have worked to address the concerns of neighbors, altering their plans to improve drainage at the bottom of a steep hill, improve traffic flow near the facility, and reduce noise, odor, and disruption from construction.

“We’ve really drilled down into a lot of detail,” Dr. Jeffrey Goldstein, Valley Green Grow’s founder, said in advance of the meeting. “Our aim is to try to not cut corners, and be very thorough and very complete.

Valley Green Grow submitted its latest revision just before the Wednesday planning board hearing, saying it had made minor tweaks in response to feedback by town contractors. But residents objected, saying they had been given no time to review the latest version, and argued the changes were submitted after a December 21 deadline.

Charlton Board of Selectmen clerk Karen Spiewak, commenting as a resident, agreed, saying “this is no way to conduct business.”

“This may be the letter of the law, but it certainly isn’t the spirit of the law,” she said.

Even if it ultimately prevails in the zoning fight, Valley Green Grow must still receive permission from Charlton’s Conservation Commission and state environmental regulators. It must also secure a marijuana business license from the Massachusetts Cannabis Control Commission.

In December, Valley Green Grow’s application to the state Department of Public Health (DPH) for a medical marijuana license expired. The cannabis commission has since assumed oversight of that program from DPH, and the firm intends to reapply. But health regulators earlier this year said that Valley Green Grow’s plan to host other licensed marijuana growers as tenants in its six proposed greenhouses was not allowed under state regulations.

Goldstein had previously tried to win approval for a similarly large growing operation in North Andover, inside a former Lucent Technologies plant he owns. But voters there voted to ban recreational marijuana facilities, effectively scuttling the project, which was slated to include the cultivation of both medical and recreational marijuana.

“It’s a shame,” Goldstein said of the opposition by Charlton residents to his project. “What they don’t realize is, living next to an apple orchard, they have pesticides and chemicals and all sorts of things flowing down into their yards... We’re much more innocuous in terms of actual impact.”

If it wins all the required approvals, Valley Green Grow has promised the town payments of up to $7 million annually, plus $500,000 up front to fund the design of a new police and fire station.