A government-watchdog group filed complaints Tuesday with the Federal Election Commission and the Justice Department asking the agencies to investigate a $150,000 payment to a former Playboy centerfold model who sold her story of an alleged affair with President Donald Trump to the publisher of the National Enquirer.

American Media Inc. agreed in August 2016 to pay Karen McDougal for the rights to her story of the alleged 2006 affair, which it then didn’t publish, a tactic known in the tabloid world as “catch and kill,” The...