Darchula, Nepal - Ramesh Sista decided he would rather drown than die of hunger in India. The 29-year-old Nepali man managed to swim across the Mahakali river, which acts as a natural border between the two countries.

"You can take me anywhere from here. Hundreds like me are stranded on the other side, sleeping like animals on the ground. This country needs to save them," Bista said on Tuesday, as he was dragged away by the Nepali police in Darchula.

Thousands of Nepalis in India started walking home after Indian Prime Minister Narendra Modi imposed a complete lockdown on March 24.

An estimated two million Nepalis live and work in India, many as low wage labourers.

On March 22, the Himalayan nation closed its open borders with India, where both people and goods flow freely, to prevent the spread of the new coronavirus.

It also banned all international flights from March 22, two days before the lockdown was imposed in its southern neighbour.

This has meant that thousands of Nepalese, who want to return home, are stuck at border points across the country's 1,700km (1,056-mile) border with India.

On the far western corner of Nepal is Darchula, a district of more than 130,000 people with more than half the population living in desperate poverty.

For many Nepalis in Darchula and the surrounding districts, their livelihoods are across the border.

But now, the foot-bridge over the roaring Mahakali river that links the two countries has been sealed, stranding about 800 Nepali workers in Dharchula - an Indian border town in Uttarakhand, a state of 10 million people.

Nepali workers near the Mahakali river which divides India and Nepal [Narendra Karki/Al Jazeera]

Nepal has just had a handful of COVID-19 cases; five confirmed by the end of March. All the cases so far were "imported" and the government says the lockdown would be important to avoid local transmission of the virus.

But experts say the low level of infections is due to fewer than 1,000 tests that have been conducted in the nation of about 30 million people.

Nepal's government built a limited number of border quarantine facilities but when the numbers started increasing, they could not cope.

Thirty-five-year-old Subash BK walked for three days and two nights to reach the Darchula border. He worked as a porter in local markets and had been in Uttarakhand for just more than a month.

"If we can go back home, we'll somehow manage to survive. But now, we've been told that we're quarantined here. I don't know for how long," he told Al Jazeera.

Human Rights Watch in a statement released on Tuesday said Nepal had abandoned its workers in the fight against COVID-19, as it denied the right to return of its citizens. A large number of Nepalese also work in Malaysia and the Gulf countries.

"Nepal's government faces huge challenges to keep its people safe during the COVID-19 pandemic, but its response should not be denying citizens the right to come home," HRW said.

"The Nepali government should act immediately so that its citizens can come home."

The Nepali authorities said the problem of migrants at border points has now been solved.

"The migrant Nepalis should have stayed put but because they did not understand the idea of lockdown, they started coming home," Yadunath Paudel, the chief district officer for Darchula, told Al Jazeera.

"But we've managed to coordinate with the Indian authorities for food and shelter in five places. We request the people to stay calm," Paudel said.

But Nepalese workers are not the only group that has fled Indian cities in the wake of the strict lockdown announced by New Delhi. Hundreds of thousands of Indian migrant workers have left cities as economic activity came to a standstill.

About 900km (559 miles) south of Darchula, approximately 1,000 Indian citizens are stuck in the Nepalese border town of Birgunj. They have not been allowed to enter India.

Ramesh Bishta crossed Nepal-India border [Narendra Karki/Al Jazeera]

Nepalese journalists say the Indian police fired warning shots at them when they tried to cross over - a charge Indian police denied. All of them have now been quarantined on Nepal's side of the border, in a local college.

Some Nepalese have sneaked into their villages through fields and rivers without notifying authorities due to the stigma of quarantine.

Al Jazeera met a man who lives near Birgunj, hiding at home. Aditya, whose name has been changed for privacy, had gone to India for work, selling scraps in a nearby town.

He walked 50km (31 miles) back home, through rivers and fields. "I'm scared. If people know I crossed the border illegally, they will mark a big cross on my house. And they will send me to live alone," he said.

Some local government authorities have started putting a red cross on homes of migrants who have recently returned. Aditya is concerned that his entire village will stigmatise him and his family as disease carriers.

He is more concerned about the "lone quarantine cell", which he believes is like jail.

People Al Jazeera spoke to in Dharchula on the Indian side said they do not want the Indian government looking after them.

"We migrate because we have to. Our crop is not enough even for a month. To buy food, we have to work in India," said Mahata from Dharchula via phone.

"If the Nepali government cannot give us work, they can at least let us come home. If we get the disease, we will sell our house and pay for our treatment. We don't need the government to pay for it."

But Mahata agrees that quarantine is a must as well. "I also have a wife and children at house. I don't want to give them a disease. I will quarantine. But please let us come back to Nepal."

Authorities in Nepal say the borders shall remain sealed, at least for another week.

Additional reporting by Kanchan Jha and Narendra Karki