In comparison, average hourly wages for workers at U.S. companies climbed 9.5 percent from the end of 2012 to 2016, according to the U.S. Bureau of Labor Statistics.

From fiscal 2012 to 2016, the median annual compensation for board members rose 19.5 percent, to $245,000, corporate board data company Equilar said Wednesday. The study analyzed the 500 largest U.S.-based companies by revenue that trade on a major U.S. stock exchange, adjusted to mirror the mix of industries included in indexes such as the S&P 500.

Directors sitting in corporate boardrooms got a pay raise twice that of the average American worker, and it's not even a full-time job.

Board directors meet regularly to discuss a company's performance, risks, financial performance and CEO compensation. They are paid a mix of cash and stock for their expertise. Companies also typically reimburse directors for travel, accommodation, food and other expenses they have in connection to their role on the board.

Many directors hold positions on more than one board and some are company executives themselves. Equilar said that since a growing part of that rising compensation also comes from a company's stock, directors may also be biased toward certain decisions.

Equilar found that Regeneron Pharmaceuticals paid its directors the most in 2016 at nearly $2.1 million. Tesla was second at nearly $1.7 million, followed by Goldman Sachs at $575,000.

Regeneron's proxy filed earlier this year says its board had six regular meetings and three special meetings in 2016, and its various committees also met between two and 10 times. The proxy said directors attended at least 75 percent of the meetings of the board and their assigned committees.

Regeneron Pharmaceuticals board pay in 2016

Source: Company filing. View table footnotes on page 32 of the proxy here.

Salesforce.com, Oracle and some pharmaceutical companies were also among the 10 companies with the highest director pay.

Overall director compensation has climbed 2 to 5 percent annually, with well over half of pay coming from stocks, according to Equilar.

That said, Alan Johnson, managing director of compensation consulting firm Johnson Associates, pointed out that the level of director pay is set in a competitive marketplace and has actually grown at a slower pace than that of company executives.

"The real issue is the income growth of the middle [class] has been sluggish," Johnson told CNBC in a phone interview. "We have to grow our economy so the middle-class American will feel better."

Inflation has grown far more quickly than average U.S. wages, but matches board of director compensation. The personal consumption expenditures index, excluding food and energy, has climbed 19.9 percent from the end of 2012 to 2016, according to the Bureau of Economic Analysis. The measure is the Federal Reserve's preferred inflation indicator.