Messaging startup Snapchat Inc. has agreed to pay more than $100 million in cash and stock to acquire Bitstrips Inc., the maker of popular virtual-sticker application Bitmoji, according to two people familiar with the deal.

The Bitmoji app lets users create customized cartoon avatars that look like them, and then send them using a special smartphone keyboard. The app, initially launched in October 2014, has surged in popularity, never falling out of the coveted rankings of the top 100 apps in Apple Inc.’s app store at any...