NEW YORK— International Business Machines Corp. is piloting its Jeopardy-winning Watson technology as a tool for catching rogue traders at large financial institutions, executives said in an interview Monday.

The company’s Watson Financial Services product looks for patterns in traders’ chats and emails while also analyzing numerical trading data. The surveillance tool is being piloted with a handful, or fewer than 10, financial-industry clients, said Bridget van Kralingen, senior vice president of IBM Industry Platforms.

The company is also developing other capabilities for the technology, Ms. van Kralingen said. One analyzes regulatory text to identify obligations that companies might face and to help assess whether the company’s compliance programs are sufficient to comply with the rules. Another would assist banks in detecting suspicious customers or transactions.

The services are a new foray for IBM as it seeks to win a larger slice of the multibillion-dollar business of helping banks comply with regulations. The project is one of many efforts to make the Watson artificial intelligence computer program bear fruit six years after its debut besting human contestants on the Jeopardy game show.

In September, IBM agreed to buy financial consultancy Promontory Financial Group, a move executives said was designed to help train Watson in the Byzantine business of bank rules.