"We believe this new transaction addresses competitive concerns previously raised with respect to the prior transaction," Pessina said in a statement Thursday. "It will allow us to expand and optimize our retail pharmacy network in key markets in the U.S." Fred's CEO Michael Bloom said in a statement Thursday that "this is a disappointing outcome." Though, he added that the termination of the transaction between Walgreens and Rite Aid has no impact on Fred's "transformation strategy" or "ability to execute." Walgreens had initially offered to buy Rite Aid for $9.5 billion in late 2015, but the chain struggled to gain approval as federal regulators worried that a merger between the two would lead to a monopoly of sorts.

Rite Aid's shares tumbled 17 percent, to $3.24, while Walgreens' shares were up about 4.4 percent, at $80.51, after the announcement.

Walgreens has also terminated a related deal to sell 865 Rite Aid stores to Fred's , which sent Fred's shares plunging 27 percent in premarket trading Thursday.

Instead, Walgreens will acquire nearly half of the smaller rival's U.S. stores for $5.18 billion in cash, something Walgreens CEO Stefano Pessina told analysts and investors is "more attractive than the transaction it replaces." Notably, Walgreens won't assume any debt from Rite Aid in this deal.

Drugstore chain Walgreens Boots Alliance on Thursday called off its deal to buy Rite Aid after struggling to win antitrust approval.

"[In today's deal] Walgreens likely focused on the stores that wouldn't be in dispute," Raymond James analyst John Ransom told CNBC's "Squawk Box" on Thursday. "For Rite Aid, this is a thermonuclear war."

Ransom said that retail pharmacy companies make up one of the most difficult sectors to cover as an analyst today because of the evolving landscape, and Rite Aid is in a position now where it needs to "retrench" and figure out how to grow its business, he added.

"I think the drug stores are going to be dragged kicking and screaming into doing some sort of home delivery," the analyst told CNBC. "Teaming up with an Uber" could be one way to go, Ransom said.

Even Walgreens CEO Pessina said Thursday that it continues to be a "challenging market for pharmacy," with those challenges continuing and possibly intensifying in the months and years to come.

Other analysts are predicting Amazon — which just announced its plans to acquire grocery chain Whole Foods — will make moves into pharmacy operations in the future.

But retail analyst Ransom said he isn't sold on that idea, yet. "I think the Amazon threat is overstated because it's hard to get into the drug business."

Walgreens said it expects the initial closing of the deal to happen within six months, and then it will begin buying the 2,186 Rite Aid stores and convert them to Walgreens. The larger drugstore chain will also pay Rite Aid a $325 million termination fee.

Rite Aid, which had nearly 4,600 stores in the U.S. as of May, said the stores included in the Walgreens deal are primarily located in the Northeast, mid-Atlantic and Southeast.

The deal also includes three Rite Aid distribution centers located in Connecticut, Philadelphia and South Carolina.

Rite Aid added that it will provide certain transition services to Walgreens for up to three years after the deal closes.

"While we believe that pursuing the merger with [Walgreens] was the right thing to do for our investors and customers, this new agreement provides a clear path forward and positions Rite Aid as a strong, independent, multi-regional drugstore chain and pharmacy benefits manager with a compelling footprint in key markets," Rite Aid CEO John Standley said in a statement.

"The transaction offers clear solutions to assist us in addressing our pharmacy margin challenges and allows us to significantly reduce debt, resulting in a strong balance sheet and improved financial flexibility moving forward," Standley added.

For Walgreens, the new deal is expected to modestly add to adjusted earnings per share in the first full year after its close and generate savings of more than $400 million.

Rite Aid said it expects to use a majority of the net proceeds from the transaction to repay existing debt.

After the completion of the transaction, Rite Aid will continue to operate EnvisionRx, its pharmacy benefit manager, RediClinic and Health Dialog subsidiaries in key communities, the chain said.

Now the question is how Rite Aid will survive as a smaller entity, GlobalData Retail managing director Neil Saunders said in an email. After the deal, Rite Aid will be struggling with "economies of scale, especially for a company that is already struggling to turn a profit even before interest payments are taken into account," he added.

"For Walgreens, the deal allows it to boost both the top and bottom lines at a time when growth is harder to come by. ... The costs of converting stores to the Walgreens format should, by and large, be neutral thanks to the improved margins Walgreens brings."

— Reuters contributed to this report.