Goldman Sachs CEO David Solomon has a plan to end the era of all-male, all-white corporate boards: The investment bank will refuse to take a company public unless it has at least one woman or non-white board member. The move could make a big difference with male-dominated startups, experts say.

Under Solomon's new rule, which goes into effect on July 1 in the U.S. and Europe, Goldman Sachs wouldn't have signed on as an underwriter for WeWork, which had a male-only board when it filed to go public last year. (Soon after, WeWork ended up pulling its IPO following investors questions about its financial losses and corporate governance.)

The push toward greater diversity comes as lawmakers and policy experts are questioning the lack of progress of women inside the boardroom and the C-suite. Even though women hold about 1 in 5 board seats in S&P 500 companies, the majority of businesses still have boards that are mostly composed of men, according to the MIT Sloan School of Management.

Get Breaking News Delivered to Your Inbox

Academic research backs up the benefits of a diverse board, with studies showing such companies make better investment decisions and scale back on aggressive risk-taking.

Yet there remains a glaring lack of board diversity among venture-backed private business, which have provided the U.S. with some of the biggest IPOs in recent years.

About 60% of the most heavily venture-backed companies lack a single woman on the board, according to a study published last month from Crunchbase, Him For Her and Northwestern University's Kellogg School of Management.

These companies, founded in the last 16 years and with at least $100 million in venture funding, provide a pipeline for underwriters like Goldman Sachs, which reap profitable fees and also offer coveted shares in the IPO to their clients.

"You have mostly male founders and mostly male VCs" in Silicon Valley, noted Lorraine Hariton, the chief executive of Catalyst, a nonprofit that focuses on women in the workplace. "The typical early stage board has 5 members, and they'll have two founders and two VCs, and maybe one outside board member, and most of them are men."

Hariton added, "They come into being public with an all male, all-white board."

She believes the Goldman Sachs decision could help change that all-male bias. "This is a big deal," she noted. "[Private companies] will check the list of things they need to do, and one is to get a diverse independent director."

Beyond tokenism

It's not enough to have one woman or one non-white board member, Hariton added. Yet "tokenism" is often the case among both public and private companies, with the Crunchbase study finding that 3 of 4 private companies without all-male boards had only one woman on the board.

"People who are the 'only one' find it difficult to speak up or be heard," Hariton said. She said it benefits a company to have at least three diverse board members, for that reason.

California's new law about gender diversity in the boardroom could also help push the needle. The measure, signed into law in 2018, requires at least one female director on the board of each California-based public corporation. But at the end of 2021, companies with six or more board members must include at least three female directors.

Goldman's new rule will require at least one diverse board member to start with, but then require two by 2021, Solomon told CNBC on Thursday. "Look, we might miss some business, but in the long run, this I think is the best advice for companies that want to drive premium returns for their shareholders over time," he said.