SAN JOSE — An arson-gutted sex shop property that was scorched by a fatal fire has been bought by a realty entrepreneur who envisions a residential tower on the downtown San Jose site.

Last September, the Craze 4 Toys Adult Superstore was destroyed in a fire that was deliberately set by the establishment’s owner, Pirtpal Singh, 33, who died in the blaze after he had been evicted from the retail space, according to arson investigators.

For months, the site at 17 E. Santa Clara St. has been boarded up. Now, plans have sprouted that would dramatically improve the property with the creation of more housing for downtown San Jose.

“What we want to do is build a residential tower there,” said Loida Kirkley, chief executive officer with Roygbiv, a real estate development firm whose affiliate has purchased the property. “We want to build it as high as we can.”

The new tower would contain about 104 units, Kirkley said. The maximum height in that part of the downtown is about 265 feet, she estimated.

The units in the new tower will be offered on a for-sale basis, rather than rentals.

“If the residents have pride of ownership in their homes because they bought them, that will help the downtown, that will help revitalize the area,” Kirkley said. “People will have a stake in what happens in their neighborhood.”

In 2007, the opening of The Axis highrise marked the last time a downtown San Jose tower featured only units that were for sale.

Acting through affiliate Downtown SJ Towers, the Kirkley-led Roygbiv firm paid $3.4 million for the property, according to Santa Clara County public documents filed on Feb. 27. The seller was Eunice Kim, a Cupertino resident, the property files show.

The purchase was arranged through brokers Erik Hallgrimson of real estate firm Cushman & Wakefield; and Raymond Rodriguez and Anthony Zizzo, both of Marcus & Millichap, a real estate firm.

The buying group financed its purchase through a $2.3 million loan from Menlo Park-based Tiantai, the county records showed.

“We really see a lot of potential in downtown San Jose,” Kirkley said. “We want to develop housing near transit. The downtown gives you the most viable options for that.”

The location near the corner of North First Street and East Santa Clara Street boasts advantages over both the next few years and well into the future.

“You have the Bank of Italy building renovation across the street, there will be a BART stop located right there,” Zizzo said. “The longer-term play is you have the Google development at Diridon Station that will bring thousands of jobs to downtown San Jose.”

The prospect of replacing an adult sex shop with a modern housing tower underscores the kinds of transformations that beckon for the urban core of the Bay Area’s largest city.

“This site is prime for redevelopment,” Rodriguez said. “Downtown San Jose is going through growth and change, and more areas are being revitalized.”