On time and slightly under budget, the builders who have been working in Birmingham on the biggest public cultural project in Britain packed up and cleared out on Monday in preparation for the librarians to move in.

"It's a strange feeling that once the books arrive I'll never see this space like this again," said Brian Gambles, the head librarian who for the last three years has also been project director for the enormous building now towering over Centenary Square in Birmingham.

Miles of bookshelves towering up on nine levels, with another below ground, are ready and waiting to receive nearly a million books.

The Shakespeare Memorial Room, complete with Victorian bookcases and decorative plasterwork, and contents including one of the world's largest Shakespeare collections of books, pamphlets and memorabilia, has already been carefully installed in the golden pillbox-shaped chamber crowning the structure, intended by the Dutch architects Mecanoo to be seen from far across the city.

Herbs and spring flowers are flourishing in the third and seventh-floor roof gardens. "Imagine taking your book and your cup of coffee and walking out into this wonderful view," Gambles said, staunchly ignoring the rain and bitterly cold wind. "This is going to be one of the best places in the city."

The library's exterior. Photograph: Christopher Thomond for the Guardian

The £188.8m project, at a time when local authority libraries all over the country are being closed or cut back, has not been without controversy. Opening hours have been reduced at the city's branch libraries, but the library budget has actually been increased, and Gambles said no closures had been made or were planned. "The community libraries are and must remain an integral part of our plan – we recognise that there will be people who will always go to their library just down the road instead of coming here, and we don't want to break that link."

By the time the city authorities had decided to demolish the 1960s library, its concrete brutalist style had become trendy again. Despite the problems with the structure, with its chaotic levels, leaking roofs, bewildering layout and dimly lit interiors, the building splendidly sited beside the handsome Victorian city hall and museum had passionate admirers.

The new library not only neighbours but will share a vast foyer with the renovated home of the acclaimed Birmingham Rep theatre company, and just beyond it is the symphony hall, making an impressive complex that Gambles hopes will create a vibrant new cultural quarter, open and full of events almost around the clock. As well as linking to the theatre, the library has its own music, exhibition and performance spaces.

No passerby could miss the new building. The levels are stacked up like a pile of children's building blocks, and wrapped in a lacy metal skin which in sunlight casts intricate shadows on the floors inside and is also intended – according to the architects – to recall the tradition of jewellery making and the city's industrial heritage. The golden box at the higher levels blocks light from the archive storeys and their precious collections, including manuscripts from the 12th century on, and an internationally renowned photography collection.

It says something about the difficulties of the old library that a special hoist had to be built to help get nearly a million books out and into the new building "There is one creaky old books lift, but we really feared it wasn't up to the job," Gambles said.

The opening is due on 3 September, and Gambles confidently expects the people who have paid for it to pour in from the start. At its height the old library had 2.2 million users: he predicts numbers will shoot up to 3.5 million, which would make it the most visited public library in Europe. "It's going to be a sprint over the next few months – but we'll get there," he said.

• This article was amended on 3 May 2013. The original said more than 2.3 million books were being transferred to the new library.