Federal investigators are probing a subsidiary of Caterpillar Inc. to determine whether it was dumping train parts into the ocean near the Port of Long Beach, Calif., as part of a possible scheme to bill railroad companies for unneeded repairs, people familiar with the situation said.

The Peoria, Ill.-based maker of heavy equipment disclosed in a securities filing three weeks ago that it had received a federal grand jury subpoena to provide documents and information on its Progress Rail unit, which repairs locomotives and railcars....