"Harvey has been a mentor to many, and his influence has made an indelible impact on generations of professionals at Goldman Sachs. I want to thank Harvey for all he's done for the firm," Blankfein said.

There was no specific reason in the Goldman announcement about why Schwartz decided to retire.

The announcement came amid heightened speculation about what happens when Blankfein steps down from the investment banking giant. A report Friday from The Wall Street Journal that Blankfein was likely to step down as soon as this year accelerated the chatter about who would take over.

"I look forward to continuing to work closely with David in building our franchise around the world, serving our expanding client base and delivering strong returns for our shareholders," Blankfein said in a statement.

The moves seem to put Solomon in direct line to succeed Lloyd Blankfein when he retires as CEO.

David Solomon will take over as president and chief operating officer at Goldman Sachs as his sole competitor for the company's CEO job, Harvey Schwartz, retires in April, the company announced Monday.

Goldman began actively considering Blankfein's successor after he was diagnosed with a treatable form of lymphoma in 2015, for he received chemotherapy and was cancer-free by October 2016. He has not indicated a time when he will step down, and there's been speculation that he could hold off until 2019 and perhaps stay on as chairman even after leaving the CEO position.

The decision was made at a Feb. 20-21 board meeting, sources told CNBC.

"The board has been intensely focused on this for a couple years," said a source familiar with the board's thinking. Blankfein will be leaving "sooner rather than later," and no one else has been seriously considered as his successor beyond Schwartz and Solomon, the source added.

Goldman shares, which have underperformed for much of Blankfein's tenure, were up 1.4 percent after the announcement to a record high. Analysts generally reacted positively to the news.

"David Solomon ran the best growing part of Goldman for the decade before assuming the COO position 15 months ago," Wells Fargo analyst Mike Mayo said in a note. "The challenge and opportunity, in our opinion, is for Goldman to better monetize its corporate CEO relationships for deeper activities with both banking and trading.

Vertical Group analyst Dick Bove, who has called repeatedly for Blankfein's ouster, said the moves announced Monday showed that the board "understands that the company was pursuing the wrong business model. ... The firm is now likely to go 'all-in' to investment banking and lending. Selecting David Solomon to lead this new charge is absolutely the right decision. He is a highly qualified banker in a firm with a superb investment banking business."

KBW analyst Brian Kleinhanzl added, "We have met with David Solomon in the past and we believe him to be a capable leader at GS with significant experience running the investment bank and a longstanding member of GS' management committee."

With trading volumes drying up and regulators still honing in on big banks, it's been a difficult time for Wall Street institutions like Goldman.

The firm is expected to refine its focus on investment banking and has been encouraged to take over small competitors to expand its base.

"Lloyd managed to steer it fine," said Christopher Whalen, head of Whalen Global Advisors. "If you look back over that period, they really didn't take huge lumps and they managed to avoid most of the risk."

Schwartz has been with Goldman for 20 years, moving from securities and investment banking up to the chief financial officer position and most recently as president and co-chief operating officer with Solomon.

Solomon, who has been with Goldman since 1999, served 10 years as co-head of the investment banking division before ascending to his current position. (Outside Goldman, he's also known as a DJ D-Sol.)

"We expect the transition to go smoothly," said Charles Peabody, an analyst at Compass Point Research. "We expect no change in strategic direction. If there is a risk, it's largely likely to be confined to cultural, and not financial or strategic, issues."

—With reporting by Andrew Ross Sorkin.