Armed robbers posing as police officers held a gun to the forehead of US gold medallist swimmer Ryan Lochte in a taxi hold-up in Rio de Janeiro, feeding concern about the safety at the first Olympic Games in South America.

Lochte was returning to the Athletes' Village from a party hosted by the French Games delegation on Sunday, with swimming teammates Gunnar Bentz, Jack Conger and Jimmy Feigen, when armed men carrying police badges pulled them over.

Lochte, however, apologised after Brazilian police said the four swimmers fabricated the story as CCTV footage showed an altercation at a petrol station.

He had earlier said the men carrying police badges told the swimmers to drop to the ground and demanded their money and belongings.

When he refused, one of the men placed the pistol against the swimmer's head, and he then handed over his wallet.

"While it is true that my team mates and I were the victims of a robbery early Sunday morning, what is most important is that we are safe and unharmed," Lochte said in a statement.

He said he was looking forward to returning to the United States and starting training with an eye to competing at the Tokyo Games in 2020.

The incident stoked worries about the safety of competitors and visitors to Rio, which has a long history of violent crime, though Brazil's sports minister insisted that athletes who remained on the Olympic site had experienced no problems.

"The security operation for these Games has been totally effective," Leonardo Picciani told reporters.

"No athlete has had problems in their place of residence, in training or in the Athletes' Village."

Police are investigating the incident and trying to locate the taxi driver involved, police sources said.

In addition to the swimmers, Swedish tourists were briefly abducted when they visited a slum, Portugal's visiting education minister was robbed at knife point, bullets flew into the equestrian centre and a Games bus was attacked with stones.