To any casual observer, the area just south of Trump Tower in Midtown Manhattan is obviously wealthy: The blocks are crowded with skyscrapers, and stores include Versace and Ferrari. Diners can pick at the foie gras and caviar on La Grenouille’s $172 prix-fixe dinner menu.

In the eyes of federal-bank regulations, though, that sliver of New York City is a poor neighborhood where median incomes are relatively low.

The...