Paris is many things: a city of romance; a hotbed of culture, and the inspiration for countless artists, musicians and poets. It's also a place that, for more than 40 years, had a special relationship with the jazz trumpeter Miles Davis. Paris was the first foreign city Davis ever visited, and it was one of the last major cities he played in, shortly before his death on 28 September 1991. Now the relationship between musician and city is celebrated in a new exhibition at the Musée de la Musique.

In 1949, a 22-year-old Davis travelled to Paris, as part of a quintet that included the pianist Tadd Dameron. The quintet was booked to play at the first Paris international jazz festival since the war ended. In the US, Davis was already a rising star in the jazz world, but while he was highly respected among his peers, in mainstream America he was seen as a second-class citizen. It was a time when segregation and discrimination were rife, and most US states enforced anti-miscegenation laws. But France was a different story, and nothing could have prepared Davis for the reception he would receive in Paris. "This was my first trip out of the country," recalled Davis in his autobiography. "It changed the way I looked at things forever ... I loved being in Paris and loved the way I was treated. Paris was where I understood that all white people were not the same; that some weren't prejudiced."

"Miles often talked about Paris," says the Australian film director Rolf de Heer, who worked with Davis in Paris in 1990. "The French were in love with Miles and treated him like a god. He liked that because it was a form of respect he didn't get in his own country." French jazz pianist René Urtreger adds: "Miles was proud and touched by the fact that in France, jazz was considered to be very important music."

The Miles Davis and Tadd Dameron quintet played at the Salle Pleyel concert hall, and Davis was soon befriended by Boris Vian, a 29-year-old French polymath, whose numerous talents included writing, poetry, engineering, songwriting and playing jazz trumpet. Vian introduced Davis to Picasso and Jean-Paul Sartre, and the group would sit together in hotels, cafes and clubs in the Saint-Germain district, using a mixture of broken French, broken English and sign language to communicate. Davis also met another acquaintance of Vian's: the actor and singer Juliette Gréco. Gréco, who was almost the same age as Davis, first met him at the Salle Pleyel: Gréco stood in the wings with Vian's wife, Michelle, watching Davis play. Gréco's long black hair, large dark eyes and petite frame soon attracted Davis. Gréco in turn, was entranced: "I caught a glimpse of Miles, in profile: a real Giacometti, with a face of great beauty," she said in a 2006 interview. They were introduced and fell in love.

Davis and Gréco would often explore Paris together; walking hand-in-hand by the banks of the Seine, drinking in cafes and listening to music in clubs. "Juliette was probably the first woman that I loved as an equal human being," recalled Davis. He also fell in love with Paris; with its smells of coffee and cologne, and with the freedom it offered.

Davis wasn't the only black American musician who fell for the charms of Paris, and some (like the drummer Kenny Clarke) decided to stay in France rather than return to a harsher life in the US. But back home, Davis had a partner and two young children. He said goodbye to Gréco (although they would remain in touch all their lives) and returned home. But back in America, jazz was in crisis, with clubs closing down and gigs becoming harder to find. Depressed by his longing for Gréco, his status in American society and deteriorating work prospects, Davis became a heroin addict. It would take him four years to kick the habit.

In 1956, a cleaned-up Davis returned to Paris for the start of a European tour featuring the Birdland All Stars (Birdland was a top New York jazz club, and other artists on the tour included saxophonist Lester Young and pianist Bud Powell). Davis played with a French rhythm trio that included Urtreger: "The 1950s were a golden age for jazz in France," says Urtreger. "In England, you had union laws that restricted the number of American jazz artists who could play there, but in France, we welcomed everybody." Davis also formed a romantic attachment with Urtreger's older sister, Jeannette. The following year, he returned to Paris for another tour and was joined again by Urtreger. Around the same time, Davis was asked by French director Louis Malle to compose the soundtrack to the film noir Ascenseur pour l'Echafaud (Lift to the Scaffold). The soundtrack, consisting of improvised music played by Davis, Urtreger and others, was a huge success.

Davis regularly returned to Paris throughout the rest of his life. In 1989, he received one of Paris's highest awards, the Grande Médaille de Vermeil, which was presented to him by the then mayor of Paris (and future French president) Jacques Chirac. Urtreger attended the ceremony: "Miles was very moved and very honoured," he says. In the same year, Davis played at the 10th annual Paris festival of jazz. The next summer, Davis spent three weeks filming in Paris, playing the role of a jazz trumpeter in the movie Dingo. "I'll never forget after we had shot the final scene." says De Heer. "We were on the banks of the Seine and Miles turned to me and said, 'I just don't want this to end,' and part of the reason was that he was so comfortable being in Paris."

On 10 July 1991, Davis played a remarkable concert in the city at the Grande Halle de la Villette. Entitled Miles and Friends, it saw the trumpeter and his current band playing with many of his old associates including saxophonists Jackie McLean and Wayne Shorter, keyboardists Chick Corea, Herbie Hancock and Joe Zawinul, and guitarists John Scofield and John McLaughlin. The English bassist Dave Holland, who played with Davis in the late 1960s, also took part: "It was like a festival based around Miles and people who had played with him," he says. The French pianist Katia Labèque met Davis in his dressing room afterwards: "He was very moved and very happy about the concert," she notes. Six days later, Davis was back in Paris to receive one of France's highest cultural awards: he became a Knight of the Legion of Honour. Davis received the award from French culture minister Jack Lang, who described him as: "The Picasso of jazz." Barely two months later, the musician was dead.

But Paris's love affair with Davis lives on. In October, a major Miles Davis exhibition opened at the Musée de la Musique. Called We Want Miles, it gathers together some 400 Davis-related items (including music scores, videos, instruments and photographs) from around the world. "Miles is the first jazz musician to be honoured in this way," says exhibition curator Vincent Bessières. He says that the number of visitors is expected to reach 50,000-70,000 – similar to the numbers who attended previous exhibitions on John Lennon and Jimi Hendrix.

The reason is simple, says Bessières: "Paris was special to Miles, and Miles was special to Paris."

We Want Miles – Miles Davis: Jazz Face to Face with Its Legend is at the Musée de la Musique, Paris, until 17 January, and in Montreal from April to August. George Cole is the author of The Last Miles: the music of Miles Davis 1980-1991 (Equinox)