The Ukraine crisis has entered a new and more dangerous phase after 11th-hour talks in London between the US secretary of state, John Kerry, and his Russian counterpart, Sergei Lavrov, broke up without resolution.

The diplomatic failure sets Russia on a collision course with the west, with Moscow ordering further military deployments on Friday and a contentious referendum in Russian-dominated Crimea set to go ahead as planned on Sunday.

The referendum, which will almost certainly result in a vote in favour of breaking away from Ukraine and union with Russia, will trigger the imposition of sanctions by the west on Monday.

During five hours of talks, Kerry pushed Lavrov to postpone the referendum. He challenged him over sudden Russian troop movements along the Ukrainian border over the last few days. But Lavrov offered nothing in the way of a concession that would have helped reduce tensions.

At a press conference afterwards, the Russian foreign minister described the talks as productive, in that both sides were able to set out their positions, but the two sides did not share "a common vision". Looking for a way to emphasise his point, Lavrov sought an international parallel: Crimea meant more to Russia than the Falklands did to Britain.

At a separate press conference, Kerry, looking tired and dejected, said he had put forward proposals from Barack Obama, but Lavrov replied that Vladimir Putin was not prepared to consider them until after the referendum.

Trying to sound emollient, Kerry repeatedly insisted he was not issuing warnings or threats to Russia. But he went on to say the US would not recognise the outcome of the referendum and that there would be costs for Russia if it went ahead.

His strongest comment came when he described the proposed referendum as "a back-door annexation" that would breach international law.

Asked about troop deployments, Kerry said: "We talked about these exercises, and we talked about the level of troops that are deployed, where they are deployed and what their purpose is, and I raised very clearly the increased anxiety that is created in the Ukraine as a consequence of this."

He suggested drawing all forces back while the negotiations took place, and said Lavrov had promised to take these proposals to Putin.

"All us would like to see actions not words," Kerry added.

Lavrov, at a separate press conference, insisted the referendum would go ahead as planned. Putin would "respect the will of the Crimean people".

Lavrov expressed concern over violence in Donetsk in eastern Ukraine overnight on Thursday. Casualties mounted from clashes between pro- and anti-Russian protesters there.

Graphic: Guardian graphics

The Russian foreign ministry said it reserved the right to intervene to protect Russian lives in Ukraine, the same language that led to its intervention in Crimea. The statement added to fears in Washington and in west European capitals that Putin may not stop with Crimea and may seek to extend influence over eastern Ukraine.

Protesters from a pro-Russian demonstration in Donetsk had fought with those from a rally "for a united Ukraine", resulting in the death of a 22-year-old man and injuryinjuries to a reported 26 people. Other reports said 28 people had been injured and that the young man had been stabbed to death.

After deploying 10,000 troops as well as heavy armour and artillery in border regions with Ukraine this week, Russia on Friday took further military steps, ordering six Sukhoi Su-27 fighter jets and three transport planes into its ally Belarus, on Ukraine's northern border. The Belarusian president, Alexander Lukashenko, had expressed concern over what he said was a potential Nato threat.

Kerry and Lavrov met at Winfield House, the residence of the US ambassador, in Regent's Park to discuss the crisis which began last month when anti-Russian protesters toppled the pro-Russian Ukrainian president Viktor Yanukovych.

Kerry pushed Lavrov to stop Russia taking steps escalating the crisis – creating "facts on the ground" – and instead open negotiations on alternative proposals for ending the standoff. One proposal was for Russian troops in Crimea to withdraw to their barracks and be replaced by troops from the pan-European Organisation for Security and Co-operation in Europe (OSCE).

Afterwards, Lavrov did not immediately embrace the OSCE proposal. "There is no need for an international structure in dealing with Russian-Ukrainian relations," he said.

Donetsk, the largely Russian-speaking city where many residents have close ties with Russia, declared a day of mourning on Friday. The city has been the site of repeated standoffs between pro- and anti-Russian demonstrators.

Ukrainian media said pro-Russian protesters attacked first, but the Russian foreign ministry and media reported that armed men had attacked peaceful pro-Russian demonstrators.

In a statement, the foreign ministry said Kiev was not in control of the situation in the country and had failed to guarantee demonstrators' safety.

"Radical far-right gangs armed with traumatic firearms and clubs, who began to arrive in the city from other regions of the country, attacked peaceful protesters who came out on the streets to express their attitude toward the destructive position of the people who call themselves the Ukrainian government," the ministry said.

The statement also hinted that Russian forces could intervene in eastern Ukraine to protect Russians there. "Russia recognises its responsibility for the lives of countrymen and fellow citizens in Ukraine and reserves the right to take people under its protection," it said.

The head of Ukraine's security service wrote on Friday on his Facebook page that four people had been detained in connection with the violence in Donetsk and that the detentions were "only the beginning".

Donetsk resident Anton Nagolyuk, who was present during the clashes, said they originated from the pro-Russian part of the demonstrations. The pro-Russian protesters had arrived at Lenin Square before the pro-Ukrainian ones, and police at first kept the two sides apart. The pro-Russian side threw eggs and firecrackers, and when the rally ended they started to beat pro-Ukrainian demonstrators, Nagolyuk said. The man who was killed was from the pro-Ukrainian side, he said.

"Some of the people definitely came from Russia to provoke people, but I don't know exactly how many of them there were," Nagolyuk said. "It's true there are many Donetsk residents among the [pro-Russian demonstrators], but it seems to me the most active ones are Russian."

According to Nagolyuk, many of his fellow residents fear Russia will invade their part of the country. "They were deliberately waiting for a death, an excuse to bring their troops and tanks into Donetsk, Lugansk and Kharkov," he said. "It seems to me almost everyone is afraid of war and Russian troops."