Last year, a public-art project in New York had some unusual stars: pigeons. Duke Riley attached LED s to homing pigeons and set them free at dusk on the Brooklyn waterfront. Using whistles and poles, he directed their movement like a conductor, orchestrating a mesmerising aerial dance above the East River called “Fly by Night” (below). The piece was a homage to the place: the Brooklyn Navy Yard once held a large fleet of pigeon carriers that relayed messages at sea. In Riley’s hands, the birds, an urban nuisance, became creatures of beauty which made us see the city with fresh eyes. It revealed forgotten layers of urban and cultural history, mining the location for meaning. No gallery could provide such a rich platform.

This autumn, an exhibition at the Museum of the City of New York, “Art in the Open,” examines how artists have responded to the city since its public-art programme was founded 50 years ago. Abstract sculptures began appearing on the streets in the 1960s, as artists looked beyond museums for locations to exhibit their work. New York had just undergone a vast physical transformation, with low buildings knocked down to create skyscrapers like the Seagram Building. A change in the planning regulations allowed architects to build taller buildings if they also designed pedestrian plazas in front, where art could fill the windswept void. John Lindsay, New York’s mayor, encouraged this development: on the brink of a social and economic crisis, New York needed a little shine.

“Fly by Night”, Duke Riley’s pigeon-inspired piece for the Brooklyn waterfront

It took a while for it to arrive. When the Art in the Parks initiative was founded in 1967 by the city’s department of parks, recreation and cultural affairs, the pieces it installed, often in apparently random places, were derided as “plop art”. What’s more, the parks themselves were often filled with litter and graffiti – hardly ideal venues for ambitious projects. When the artists Christo and Jeanne-Claude first proposed “The Gates” in 1979, for instance, New York turned them down. Central Park, which they envisioned swathed in saffron-coloured fabric, was just too derelict. But over the years New York cleaned up, crime dropped and Manhattan gentrified. During the mayoralty of Michael Bloomberg, who promoted the arts enthusiastically, the city hosted 500 temporary public-art projects. These included “The Gates”, which was finally installed in Central Park in 2005.

Today New York has more public art than any city in the world. The Parks Department alone now installs one artwork a week, and transportation agencies and neighbourhood groups have instituted their own programmes. Art has appeared on waterways, on traffic barriers and in tunnels. This year began with the unveiling of colourful mosaics by Chuck Close in the Second Avenue subway. It ends with “Good Fences Make Good Neighbours” by Ai Weiwei (pictured top), which consists of 300 sculptures and installations spread around the city. Like most of contemporary public art, both pieces have been designed for specific locations. One of the most powerful recent examples was Kara Walker’s “A Subtlety” in Brooklyn’s Domino Sugar Factory, commissioned by Creative Time, a public-art group. Before the building was demolished to make room for apartments, she filled it with a giant, sphinxlike female slave coated in sugar. The cavernous space reeked of molasses. “A Subtlety” encouraged us to consider the human cost of the trade and industry that powered the city’s growth.

“Vessel” by Thomas Heatherwick, which stands at the centre of the Hudson Yards development in Manhattan

While the popularity of public art has been powered by the ingenuity of artists, there has always been something in it for the city. Bloomberg recognised that bold public art could drive investment and tourism. “The Gates” drew 4m people, and these tourists generated over $250m. Property developers have courted artists in order to give their projects cachet. They have been inspired by Tony Rosenthal’s “Astor Place Cube”, an early example of interative public art and one of the most popular outdoor sculptures in the city. When this swivelling steel box was installed in 1967, the intersection where it stands was forlorn. Yet people gravitated to it, and a concrete traffic island became a destination. Thomas Heatherwick's “Vessel”, currently under construction, will try to replicate its success. This enormous, copper-coloured interactive sculpture composed of 154 staircases will sit at the heart of Hudson Yards, an area on Manhattan’s far west side which is being transformed by the largest public development in American history. Its aesthetic merit is debatable, but that’s only part of the point. The developers are more interested in whether the piece will draw people to Hudson Yards and keep them there. A structure which offers spectacle, sweeping views and a place to exercise seems well designed to do just that.

As the amount of public artwork has surged in the last decade, most of it has been in Manhattan. But the current mayor, Bill de Blasio, wants to broaden its reach. For his exhibition, organised by the Public Art Fund, Ai Weiwei has made work for all five boroughs. Highland Park, a sprawling district on the border of Brooklyn and Queens, had never presented art until Daniele Frazier, artist from the local area, made a series of giant fabric flowers there this summer. Locals, who were surprised to see them, asked why the flowers were there. Frazier explained that they existed for no other reason than to evoke wonder. For all of public art’s civic virtues, that’s enough.

Art in the Open: Fifty Years of Public Art in New York Museum of the City of New York, New York City until May 13th