The number of consumers opting not to subscribe to pay TV services accelerated in the first quarter, driven by overall losses in satellite TV customers, sluggish growth for telco TV and a strong rise in household formation, according to MoffettNathanson principal and senior analyst Craig Moffett.

Pay TV shed about 31,000 customers in the first quarter, compared to a gain of 271,000 subscribers in the same period in 2014. Cable losses actually improved in the period – basic video customers declined about 2.3% vs. a 2.8% decline in 2014 – but a loss of 134,000 net subscribers at Dish Network wasn’t enough to offset a 60,000 quarterly gain at DirecTV. Telco TV providers added 129,000 new customers in the first quarter, about half the 259,000 added in Q1 2014, and not enough to make up the difference in the other sectors.

Moffett said the rate of decline – about 0.5% over the past 12 months – may not seem dramatic, but it is the “fastest rate of decline on record and it represents by far the largest sequential acceleration we have seen to date.”

In a note to clients, Moffett said the numbers are even more disheartening if you compare them to a strong gain in new households (1.3 million) in the fourth quarter. While the U.S. Census Bureau estimated that new household formation declined by 407,000 in the first quarter, the strong fourth quarter gains and a general firming of household formation for most of 2014 should have led to a gain in pay TV homes.

Moffett added that the first quarter results also don’t include the impact from new over-the-top services like Sling TV, Sony PlayStation Vue and HBO Now because they either were introduced late in the quarter or after the period ended. And Apple, which has hinted at plans for its own over-the-top service, hasn’t fully entered the picture yet.

“It’s too soon to panic,” Moffett wrote. “But it’s not too soon to be genuinely worried.”