As of Bitcoin Core 0.14.0, these are the ways a transaction can leave the mempool:

The transaction was included in a block.

The transaction or one of its unconfirmed ancestors conflicts with a transaction that was included in a block.

The transaction was replaced by a newer version (see BIP 125).

The transaction was at the bottom of the mempool (when sorted by fee per size), the mempool had reached its size limit (see the -maxmempool option), and a new higher-fee transaction was accepted, evicting the bottom.

option), and a new higher-fee transaction was accepted, evicting the bottom. The transaction expired by timeout (by default 14 days after entering).

Also, transactions that have left the mempool can always enter them again. For example, when they're part of the local wallet and get rebroadcast, or when they're received again over the network. Eviction does not imply a transaction is cancelled.

Since Bitcoin Core 0.14.0, the mempool is saved to disk, so it persists across restarts. In earlier versions a restart would also result in a wiped mempool.