In 1917, in the depths of the first world war, the German architect Bruno Taut made drawings of his feverish architectural dreams, of crystal halls built on the tops of the Alps, in which there would be nothing but silence and a little beautiful music. He knew that they would be tricky to build, but hoped – in vain – that the monstrous expenditure devoted to conflict could be directed their way. A century later the Swiss architects Herzog & de Meuron, authors of Tate Modern and the Beijing Olympic stadium, have come closer than anyone might have expected to realising Taut’s fantasy. They have put a crystalline palace for music in the air, albeit on top of a large warehouse in Hamburg rather than a mountain, and with a hotel, apartments and car park added to the brief, at the cost of an effort only slightly less than Taut imagined.

At times you want the exceptionally exceptional exceptionalness of almost everything to ease up

The Elbphilharmonie, to give it its proper name, offers a sequence of visual and architectural otherworldliness that starts with distant glimpses from the centre of the city and doesn’t stop when you reach your seat in one of its three auditoria, of 2,100, 550 and 150 seats. Its exterior, 110 metres high, is glassy and opalescent, curvy on top, above the brick block of the warehouse, a crystal on top of a rock, teetering because bigger than its base, but also lighter looking. Alternatively – and if metaphors are mixed it’s because the building provokes it – it is a cloud on a cliff, a sail, a wave, a ship, an iceberg, a tent. The main auditorium is both bubble and cave, its surfaces unified with a wrinkled, variegated, fascinatingly repellent surface that its architects call “white skin” but is really greyish and not-human, more pachydermic, dinosaurian or alien, or rather, as it’s hard, like moon rock.

Facebook Twitter Pinterest The inviting curved stairs and rippling glass. Photograph: Herzog & de Meuron / Iwan Baan

Between the crystal and the cave is a third element, 37 metres above ground, on the layer where the warehouse stops and the concert hall starts, a “plaza” where Hamburg’s citizens can go whether or not they have a ticket to an event, and take possession of a glorious panorama of their city, with its river and docks, a horizontal sweep punctuated by spires, cranes and steaming chimneys. Here your feet are planted on brick, as if you were still at the level of the quays outside, but the white ceiling curves itself into vaults, across which constellations of light gather, suggesting a saloon, a drawing room, or an old-fashioned foyer to a musical space. The views, and the bright autumn sunlight, are caught, refracted and multiplied by reflective surfaces and curving glass walls.

Facebook Twitter Pinterest The curving escalator. Photograph: Herzog & de Meuron / Iwan Baan

The main parts of the complex are connected by spaces in which dark alternates with light, concave elements with convex and closing-in with opening-up, and the pursuit of extraordinariness is occasionally paused by something quieter. There is a studiedly low-key entrance, then an escalator that curves as if going over the brow of a hill, then, through a frameless and imperceptible wall of glass, an arresting view. Next a sharp turn left to a shallow, not-curved escalator, which takes you to the raised “plaza”. From here you can rise via inviting curved stairs into multi-level, multi-angled foyers – still pierced by views out – until, finally, you reach the strange-skinned auditorium.

Facebook Twitter Pinterest Herzog and de Meuron’s ‘vineyard’-style concert hall. Photograph: Herzog & de Meuron/Iwan Baan

In the bottom half of the journey you feel as if you are boring through brickwork, until the space unfurls into shapely, light-filled whiteness. The curve on the escalator redeems this most banal form of motion, redolent as it is of shopping malls, by imparting a sense of mystery about its destination (and no, it hasn’t been done before, laughs one of the architects delightedly, why would you want to curve an escalator?). Views, which architects so often present blandly, as if on a screen, are here wrapped into the spaces of the building, given and taken, and captured with unexpected slivers of reflection. You might call the curves and irregularities organic, but it’s spookier than that. The architects like the unease that comes from natural-looking things that are artificially made. They take a sophisticated pleasure in flirting with ugliness and kitsch.

Facebook Twitter Pinterest The Elbphilharmonie’s sound-modulating ‘white skin’. Photograph: Christian Charisius/EPA

All of which virtuosity will be a little pointless if the halls don’t sound good. The main auditorium is in the “vineyard” form pioneered in the Berlin Philharmonie of 1963, which arranges audiences in terraces around a stage more central than in the tried and tested “shoebox” form of, for example, the Musikverein in Vienna. The vineyard allows large audiences to feel closer to the performers, but is acoustically more challenging, so the architects have worked with the renowned acoustician Yasuhisa Toyota, and with the full panoply of computer modelling techniques, to make it work. The texture of the moon rock, digitally manipulated, helps to modulate the sound.

The question of its acoustic success can’t be answered conclusively until there have been a few performances to full houses, and as my ears are made of cloth I’d be the wrong person to ask. I can only pass on the not-impartial reports of the architects’ and the Elbphilharmonie’s representatives. In testing, they say, musicians were moved almost to tears by its quality. It pulls off the rare trick, they also say, of sounding warm and precise at the same time. What I can say is that it feels like a place which, when full of people, will create a powerful rapport between performers and audience.

Another large question concerns the cost as, from the point of view of the citizens of Hamburg, they may as well have sent space-freighters to ferry real moon rock. Its total cost rose from €77m, an implausibly low figure given by the city authorities when the hall was first mooted, to a final bill (with substantial additions to the brief, but even so) of €860m. The inherent cost of the concept was then multiplied by a fraught relationship between city, contractor and architects which, until a new contractual arrangement was worked out, caused the project to be suspended for two years.

It has made for what I’m told is a “rollercoaster” relationship between the project and the people of Hamburg, as up and down as the hall’s roof, ever since 2003, when Herzog & de Meuron’s concept was put forward by Alexander Gérard, a property developer who owned the warehouse. The first public and press reactions were enthusiastic, even though there wasn’t an absolutely overwhelming need for a new concert hall – WE WANT IT said one headline – so the city adopted the project and eventually paid for most of it. The association of the city’s architects wrote a letter saying that they liked Herzog & de Meuron’s concept so much that they didn’t mind if the city waived the usual requirement to hold a competition to choose the architects. Since then, the agonies of budget and contract changed the public mood in a way that may now be changing again. The legality of the architects’ appointment was also challenged, until the European court decided in their favour.

Facebook Twitter Pinterest The view from the stage… Photograph: Fabian Bimmer/Reuters

There are shades of London’s Garden Bridge project (yelp!) in the way that a private individual’s dreamed-up scheme was adopted by a public authority, including the alleged bending of competition procedures, although in Hamburg there seems to have been a more full and open public discussion early on. It also resembles the Sydney Opera House, also waterside and sail/wave-roofed and many times over budget. What matters now is whether it will experience the Sydney Opera House redemption (which the Garden Bridge will also need, if it is built), which is that the building will become so loved, and such an indispensable part of the city’s image and self-image, that its birth pains will be forgotten and forgiven.

My guess is that it will. From the point of view of an architecture critic there are some quibbles – it would be better without some angled columns that are sadly necessary to hold up the auditorium, and the reworked warehouse base does not feel as substantial as you want it to be. At times you want the exceptionally exceptional exceptionalness of almost everything to ease up, so you can appreciate just how exceptional it is. But, for its sustained imagination and invention, the skill of its making, the amazement of its best spaces and its public spirit, it is hard to beat.