The Obama administration has been quietly putting pressure on Saudi Arabia to allow women to drive, according to leaked US embassy cables. But the jailing of a woman protester, Manal al-Sharif, after she posted online a video of herself at the wheel of a car in Khobar reveals the extent of the US diplomatic failure.

The cables, part of the treasure trove allegedly given to WikiLeaks by the US soldier Bradley Manning, reveal previously unreported clashes over women's rights.

Dispatches from Riyadh describe Saudi Arabia as "the world's largest women's prison". Those words are a quote from Wajeha al-Huwaider, a female campaigner with whom US diplomats have been in contact. She posted a video on YouTube in 2008 of herself driving. Claiming millions of Saudi women were prisoners in their homes, she challenged male control over work and travel.

She regularly tried to take a taxi to neighbouring Bahrain. According to the cables, "al-Huwaider is divorced which means under Saudi law her ex-husband or her father or a brother would need to give her permission to leave the country. Although she holds a valid passport, every time she tries to leave ... she is stopped at the border to Bahrain and turned around."

The billionaire tycoon Prince Waleed, a Saudi royal, assured a visiting Democratic congressman in July 2009 that King Abdullah did support women's rights, the embassy noted optimistically. The driving ban was reportedly about to be overturned.

Speaking at his 99-storey Kingdom Tower in Riyadh, Waleed said the ban was merely a "demeaning" tribal custom and that he "relished relating his run-ins with the kingdom's religious conservatives. He was involved with the first public showings of films in the kingdom in many years. His wife has openly requested that women be allowed to drive. He supports French president Sarkozy's campaign against women wearing coverings hiding their faces."

Abdullah appointed the country's first woman deputy minister in 2009 and opened "with much fanfare" a mixed-sex science university, in front of foreign dignitaries including Prince Andrew.

The embassy noted approvingly "several subtle, symbolic gestures ... Saudi men and women, many of whom did not wear the face-covering niqab, mingled freely with international attendees throughout the ceremony. Male and female students stood side by side on stage for an emotive reading of a poem. The ceremony was interspaced with a movie showing (uncovered) young girls and boys studying together."

But there was an immediate backlash. Saad Nasser al-Shithri, a cleric from the council of senior scholars, appeared on the Saudi religious TV channel to defy the king. He denounced "mixing of the sexes" and "the teaching of deviant ideas such as evolution".

Abdullah was forced to sack him, but embassy contacts warned privately that Shithri was being regarded as a hero by unemployed young Saudis, who resented foreign students getting advantages, and by reactionary clerics, who feared a plot to impose western values.

Another cleric, Sheikh Salman al-Duwaysh, publicly attacked "mixing with women on the basis of claiming to educate them and to open the field for them to undertake jobs for which they were not created". He said such women had "abandoned their basic duties such as housekeeping, bringing up children ... and replaced this by beautifying themselves and wantonness".

The embassy was refused consent for a US "rhythm and oratory duo" called Teasley and Williams to play to a mixed audience at the university.

But the duo did appear at the Riyadh literary society before "an unprecedented mixed-gender audience (mixed by Saudi standards – the handful of women who attended sat in a screened-off block of seats across the aisle from the men). Nonetheless, the fact that women were even invited to a musical performance with men in Riyadh is remarkable".

Obama's envoy, Richard Erdman, privately scolded Saudi ministers to little effect. He "pointedly" told the notoriously reactionary interior minister, Prince Naif, that "no nation could prosper without the intellectual contributions and talent of all its citizens ... (ie women)". He said the same to the deputy foreign minister, who responded wryly that "customs were a hard nut to crack".

In a dispatch headed "Women need not apply", US diplomats recorded that US-educated Prince Mansur, the minister of municipal affairs, firmly rejected the notion that political development required the participation of women, saying issues such as women driving were "not fundamental to our society".

According to the US diplomats, the driving ban is in fact something of a charade which "dates from a 1991 fatwa issued by the late grand mufti of Saudi Arabia, Sheikh Abdulaziz bin Baz. The grand mufti claimed that allowing women to drive would result in public 'mixing' of men and women, put women into dangerous situations because they could be alone in cars, and therefore result in social chaos."

The cable continued: "Women drive anyway: there are, in fact, many instances in which Saudi women defy the prohibition.

"Women drive on private property such as desert farms or residential compounds beyond reach of police.

"Embassy contacts and media report that in rural areas women routinely drive out of necessity, without being stopped.

"Al-Hayat newspaper reported 16 February ... a woman driving in some Saudi villages is considered normal."