FRANKLIN, Tenn.—Spies are infiltrating Wall Street.

A wave of companies with ties to the intelligence community is winning over the world of finance, with banks and hedge funds putting the firms’ terrorist-tracking tools to work rooting out employee misconduct before it leads to fines or worse.

“Both Wall Street and the intelligence world...