Square Inc. is looking to get into the banking business.

The San Francisco-based finance firm led by Jack Dorsey plans to submit an application on Thursday to form a wholly owned bank based in Utah, the company said. The unit, to be called Square Financial Services Inc. would offer loans and deposit accounts to small businesses and be capitalized with $56 million in cash.

Square...