Even where a commitment to gender parity exists, companies must be more proactive to keep women on strong career paths from the lowest to the highest organizational levels, some of the top business leaders in the U.S. said Tuesday.

“We have to get the pipeline going right at the very beginning,” Mary Barra, the chief executive of General Motors Co., said at an event in New York hosted by The Wall Street Journal about improving the representation of women in corporate America.

Sheryl...