Judging by the criticism thrown at the Muslim centre planned for downtown Manhattan, you would think developers intended to build an Islamic citadel right on top of Ground Zero with "sponsored by al-Qaida" written on its front.

In fact, the proposed scheme for the much-slated "Ground Zero mosque" is neither a mosque nor at Ground Zero – it is a multifaith community centre with an Islamic prayer area, two blocks north of the site where the twin towers once stood. Now, conceptual drawings of the building have been released, revealing a planned structure that is strikingly modern and in keeping with the spirit of New York's most cutting-edge design.

Park51, named after its location on the site of an old coat factory in Park Place, would be a sleek 15-storey tower sandwiched between older buildings.

The most vivid element of the renderings, drawn by a New York-based design studio, Soma Architects, is the building's white frontage, which is broken up into a lattice of interlocking geometric shapes. At night, it would be lit up like a glistening honeycomb.

The device is a clear allusion to the intricate arabesque motifs found in Islamic architecture, and is reminiscent of the Institut du Monde Arabe in Paris, which shot the architect Jean Nouvel to fame in the 1980s. But it also pays homage to other religions, with the Jewish Star of David being clearly visible among its patterns.

Contrary to the mass of bad publicity that has been heaped on it in recent weeks, the building is designed to be multifaith and also secular.

Sharif el-Gamal, the developer of the $120m (£76m) project, told Associated Press that nearly a quarter of its space would be devoted to a sports and fitness centre, which, it is hoped, would attract New York residents of any faith and ethnicity for a fee of up to $2,700 a year per family. Another floor would be given over to a playground and childcare area.

In addition, there would be a restaurant and exhibition space, and on the 12th floor a memorial and sanctuary remembering the events of 11 September 2001 that would again be multidenominational.

"I don't think that once this thing gets built, anyone will be picketing," Gamal told AP.

The most controversial aspect of the scheme – its Muslim prayer space, which would occupy two floors in the basement – will not be a mosque at all as the construction of the building does not satisfy the stringent requirements for a sanctified mosque.

Seen from the inside, the renderings show the lattice work casting intricate shadows across the white floors, another allusion to arabesque design with its emphasis on naturally lit interiors.

The drawings are only a vision of how the building might go. An official architect for the project has yet to be appointed, and ground breaking on the construction will not begin for at least three years.

By then, the developer and the charity behind the project, the Cordoba Initiative, hope that the furore will have died down.

Gamal said that he regretted the way things had gone. "I would have done things a lot differently during this process if I understood what we were up against. People have been calling this the 'Ground Zero mosque'. It's not at Ground Zero and it's not a mosque. Our identity has been stolen from us. It has been stolen by extremists."

The attack has been led by Christian evangelical and rightwing political groups, who say Park51 is a "victory mosque", revelling in the glory of 9/11. In the most extreme case, this September, Pastor Terry Jones threatened to burn 200 Qur'ans in protest at the centre.

Could a building be less militantly Islamic?

First illustrations of the 15-storey Park51 development planned to house the Islamic centre reveal a decidedly upbeat and glamorous building, more festive than threatening. These preliminary renderings by Soma, however, should be thought of as what might be rather than what definitely will. An architect has yet to be officially appointed.

Even so, the interiors – a honeycomb structure awash with daylight – look as if they could be uplifting and special. But then, you would expect something glamorous and fresh from Soma, a New York firm founded by the Lebanese-born Michel Abboud, who has recently completed the enjoyably hip Tartinery Nolita brasserie in SoHo.

Equally, at least four floors of Park51 are to be given over to a gym and spa, and only two basement floors to a Muslim prayer hall. With a restaurant, artists' studios and a childcare centre, this $140m building clad in an abstract play of Islamic patterns could hardly be less militantly hardcore. As one young man attending the nearby Sufi al-Farah mosque in lower Manhattan told the New York Times: "Because this is a new country, it revolutionises everything. Food, industry, philosophy and even religion." He might have added: "Islamic architecture, too."

Jonathan Glancey