In a recent paper, he and his fellow researchers describe an algorithm they created that is meant to help determine whether an agency adjusted its final rule in reaction to public comments. The algorithm, he says, uncovers key word clusters to measure “regulatory sentiment” as pro-regulation, anti-regulation or neutral, on a scale from -1 to +1, with zero being neutral.

Image Credit... Michael Waraksa

If the number assigned to a final rule is different from the proposed one and closer to the number assigned to all the public comments, then it can be inferred that the agency has taken the public’s views into account, he says.

Using the algorithm, the researchers analyzed about 60,000 public comments involving 104 proposed and 67 final rules at the Commodity Futures Trading Commission from January 2010 to September 2013. That’s when a raft of new rules was proposed in response to the financial crisis. The researchers found that the agency very often heeded the public’s views when writing its final rules.

But wait — Professor Kirilenko happens to have been the chief economist for the Commodity Futures Trading Commission from December 2010 to December 2012. Could he perhaps have designed an algorithm that makes him and his former agency look good? He says that the algorithm runs on its own without human tinkering, and that he used the C.F.T.C. data because he was most familiar with it. The algorithm, called RegRank, is meant to serve as a prototype for evaluating the regulations of all federal government agencies, he says.

But a big problem exists with focusing broadly on public comments, says Dennis Kelleher, chief executive of the nonprofit Better Markets, which promotes the public interest in financial markets. An overwhelming majority of comments are “filed directly by the industry, its trade groups and its purchased mouthpieces,” he says. So if an algorithm indicates that regulators have listened to the public, he says, it really means that they are heeding an industry that wants to protect its profits and bonuses. And he says the wording of many financial regulations is so technical that average citizens may not have the ability to comment on them — even though the rules may affect them directly.