IRVINE, Calif.—Tax cheats, beware: The machines are watching.

Governments are increasingly relying on machine learning and data analytics to analyze troves of data as they seek to detect tax evasion, respond to taxpayers’ questions and make themselves more efficient.

In Brazil, the customs agency’s system for detecting anomalies now prompts...