WASHINGTON—One of the top U.S. banking regulators will issue a formal call this week for officials to start crafting a new framework aimed at governing the rapidly growing financial-technology sector—a nod to complaints from traditional banks and startups alike that current rules both stifle innovation and provide insufficient oversight for new forms of finance.

“What we’re talking about now is receptivity to exploring these issues and understanding these different models as opposed to saying ‘no’ right off the bat,” Amy Friend,...