The Kushner family has closed on a deal to unload its office tower at 666 Fifth Avenue, a building it bought at the top of the market that was ailing financially.

In a larger-than-expected deal, the New York real-estate family has agreed to lease the building to Brookfield Asset Management for 99 years, according to a Brookfield statement that confirmed earlier reporting by The Wall Street Journal.

The...