Caltrain's average weekday ridership in August decreased 2.7 percent from the same month in 2015. Year-over-year ridership declined 3.3 percent in September and 3.5 percent in October. And according to data to be presented at Caltrain's Board of Directors meeting on Thursday, average weekday ridership dropped 1.8 percent in November compared to the same month in 2015.

"We might be victims of our own success," Reisman said.

Translation: Caltrain is now known for having very crowded trains during the morning and evening commutes. That congestion is turning some riders away.

The agency wants to find a way to bring riders back, but it has to do so in the face of growing competition from ride-service companies like Lyft and Uber.

"We are a little concerned," said Reisman.

He says the train agency wants to make off-peak travel more attractive to its riders. In the coming months, the district plans to try to draw more very early morning commuters.

In April Caltrain will add six stops on its northbound weekday Train 305, a San Francisco-bound "baby bullet" run that leaves San Jose's Diridon Station at 5:45 a.m.

"We're trying to find ways to make the commute more relaxing and less crowded as a way of bringing back the passenger numbers," Reisman said.

He emphasized, though, that the long-term solution is coming: the electrification of the system. When that happens, expected in early 2021, Caltrain plans to have more trains per hour and greatly expand capacity.

The Peninsula train agency is not the only local transit system experiencing a drop in ridership.

BART saw year-over-year ridership declines of 1 to 2 percent.

From August, September, October through December, BART's weekday ridership dropped about 1 percent to 2 percent each month compared to the same period in 2015.

Where are the riders going? It's not easy to say. But in the case of BART, some riders turned off by chronic commute-hour overcrowding might be driving across the bay instead of riding the train.

Between July and December, there was a small increase in crossings over all of the Bay Area's seven state-owned bridges compared to the same time last year, according to statistics from the Metropolitan Transportation Commission.

Between July and December, the Metropolitan Transportation Commission says, the Bay Bridge saw a 1.3 percent increase in crossings, the Dumbarton Bridge increased by 1.2 percent and the San Mateo Bridge increased by 2.7 percent.

KQED's Nina Thorsen contributed to this report.