How ugly was the breakup between Facebook Inc. and the two founders of WhatsApp, its biggest acquisition? The creators of the popular messaging service are walking away leaving about $1.3 billion on the table.

The expensive exit caps a long-simmering dispute about how to wring more revenue out of WhatsApp, according to people familiar with the matter. Facebook has remained committed to its ad-based business model amid criticism, even as Facebook Chief Executive Mark Zuckerberg has had to defend the company before American...