Last October, after spending almost a year examining trends among Facebook users, a senior data scientist at the company named Tom Cunningham painted a bleak picture of one possible future for its flagship Facebook app.

At the time, Facebook noticed that users of the app—known internally as the “blue app” because of the color of its icon—were increasingly sharing on Instagram and WhatsApp, which are also owned by Facebook. In confidential research Mr. Cunningham prepared for Facebook CEO Mark Zuckerberg, parts of which were obtained by The Information, he warned that if enough users started posting on Instagram or WhatsApp instead of Facebook, the blue app could enter a self-sustaining decline in usage that would be difficult to undo. Although such “tipping points” are difficult to predict, he wrote, they should be Facebook’s biggest concern.

The research came as Facebook was beginning early work to unite the messaging systems behind the company's apps. While Mr. Zuckerberg has said publicly that the plan will give users more choice by letting them communicate across apps, the move could also alleviate growth concerns for Facebook’s core service and further cement the company’s grip on the social media landscape.