Tens of thousands of tourists, pilgrims and workers have begun leaving the disputed region of Kashmir after local officials issued a security alert and India said it had killed at least five militants who were trying to attack its forces.

The Foreign Office on Saturday issued new advice to avoid all travel to Jammu and Kashmir, adding: “There is a risk of unpredictable violence, including bombings, grenade attacks, shootings and kidnapping.”

Germany and Australia gave similar warnings to their citizens.

Indian security officials ordered people to leave the Kashmir valley after they had found evidence of attacks planned by what they said were Pakistani military-backed militants on a major Hindu pilgrimage, the Amarnath Yatra – which ends in mid-August – in Muslim-majority Kashmir.

Officials called off the pilgrimage and asked the pilgrims and tourists to return home, causing panic as visitors scrambled to arrange transport. A separate government notice also advised hundreds of students from other Indian states to leave the Himalayan valley.

Anxious visitors, including some foreigners, flooded the airport at the main city Srinagar on Saturday, many without tickets for flights that day.

“Passengers who were scheduled to return in coming days have turned up in panic at the airport today,” said the manager of one airline operating the Delhi-Srinagar route. “It’s chaotic and not many will manage seats unless there are additional flights.”

Hundreds of Indian students from outside Kashmir were evacuated in buses. “All the non-local students have left the campus for their respective states,” an administrative official at the National Institute of Technology in Srinagar told AFP.

Kashmiri residents queue outside petrol stations after the alert. Photograph: Farooq Khan/EPA

Srinagar-based Indian defence spokesman Colonel Rajesh Kalia said there had been a number of attempts by Pakistan-based militants to disturb the peace in the region and target pilgrims. Five to seven militants were killed when they tried to attack Indian troops, Kalia said, adding that arms and ammunition were recovered in the operation.

A Pakistani defence spokesperson dismissed India’s assertions as “mere propaganda”, calling them “blatant lies”.

A senior local government official in Kashmir said most of the 20,000 Hindu pilgrims and Indian tourists, and the more than 200,000 labourers, were leaving the region.

Visitor numbers have been boosted by the Amarnath Yatra pilgrimage in recent years, with more than 300,000 Hindus devotees visiting the ice stalagmite cave shrine in the Himalayas. A huge security force had been guarding the route even before the alert, and a second smaller pilgrimage, the Machail Mata Yatra, in Jammu region was also cancelled on Saturday.

Kashmiri residents formed long lines outside petrol stations, food stores and bank cash machines on Friday night after the alert was announced. But the queues eased on Saturday.

India and Pakistan have fought two of their three wars over the disputed territory since the partition of India in 1947, which created Pakistan.

The Indian government has admitted that 10,000 extra troops were sent to Kashmir – where it already has 500,000 forces stationed – a week ago. Media reports on Friday said a further 25,000 troops had been ordered there. The government has declined to say how many are in the new reinforcements.

While military authorities and the state government highlighted the security risk, Kashmir politicians have raised fears that the extra Indian troops are a sign that the Hindu nationalist government could carry out a threat to scrap Kashmir’s special status under the constitution.

• This article was amended on 5 August 2019 to remove an erroneous reference to Kashmir being “divided” between India and Pakistan in 1947.