Canadian Pacific Railway Ltd. approached CSX Corp. about a combination that would unite two of North America’s largest railroad operators, said people briefed on the matter, as the boom in North American energy reshapes the railroad industry.

CSX rebuffed the overture, made in the past week, the people said, and it isn’t clear whether CP will persist.

But...