South Korea has retracted claims that North Korea is preparing to conduct a fourth nuclear test, hours after the unification minister in Seoul said there were signs of increased activity at the North's main testing site.

But as officials in Seoul dampened speculation over a nuclear test, North Korea said it would withdraw its workers from the Kaesong industrial zone, a venture with the South located north of the heavily armed border separating the countries.

The South's defence ministry said the reported movements around the Punggye-ri site were routine and did not signal an imminent test. "We found there had been no unusual movements that indicated it wanted to carry out a nuclear test," a defence ministry spokesman said.

Earlier on Monday, the South Korean unification minister, Ryoo Kihl-jae, was reported as having told a meeting of MPs that Seoul had acquired evidence of increased movement of vehicles and personnel at Punggye-ri.

Later the same day, a "startled" Ryoo said he has "misspoken" and could not remember making the comments, which were recorded on video. A ministry spokesman said Ryoo had meant to say that North Korea had long been prepared to conduct a nuclear test, but that there were no signs of unusual activity.

Announcing the suspension of operations at Kaesong, Kim Yang-gon, secretary of the ruling party's central committee, said the regime would suspend operations in the zone and decide whether to resume work there or close it down.

"It has become impossible to operate the zone as usual due to the South Korean warmongers' reckless acts," the official KCNA news agency quoted him as saying.

North Korea last week banned hundreds of South Korean managers from entering the complex, which earns the communist state about $86m (£56m) a year. More than 120 firms from the South employ 54,000 North Korean workers.

Analysts had expected operations at the plant to wind down because of the ban and the dwindling supply of raw materials from the South.

The nuclear testing site on North Korea's north-east coast is where the country's three previous controlled nuclear detonations – in 2006, 2009 and on 12 February this year – took place.

Ryoo's comments came after a South Korean newspaper quoted an unnamed government source as saying that workers appeared to be preparing for another test at Punggye-ri.

The JoongAng Ilbo quoted the source as saying that South Korean intelligence had detected "increased activity of labour forces and vehicles" at the site. "We are closely monitoring the ongoing situation, which is very similar to the situation ahead of the third nuclear test. We are trying to figure out whether it is a genuine preparation for a nuclear test or just a ploy to heap more pressure on us and the US," it said.

New Focus International, a website run by North Korean exiles, suggested a simple mistranslation from Korean to English might have contributed to the confusion. The nuanced meaning of the Korean word for "early symptom" or "signal", the site said, did not necessarily indicate an imminent action, but referred to moves to leave open the possibility of a nuclear test.

The prospect of a nuclear test is certain to add to tensions on the peninsula amid reports that the regime in Pyongyang could be planning to launch a medium-range missile from its east coast, possibly as early as Wednesday.

Any missile launch is expected to be a test rather than a targeted strike and could take place in the runup to the 101st anniversary of the birth of North Korea's founder, Kim Il-sung, on 15 April. Another significant date is the anniversary of the foundation of the Korean people's army 10 days later.

Last week, North Korea warned foreign embassies in Pyongyang it could not guarantee their safety from the threat of conflict after this Wednesday and advised them to consider pulling their staff out of the capital.

The North has unleashed a barrage of threats since the UN imposed sanctions in response to February's nuclear test. It has also been angered by military exercises involving South Korea and the US that are due to run to the end of the month.

The regime has threatened to launch nuclear strikes against the US mainland and its bases in the Pacific, declared itself in a "state of war" with South Korea and cut its military hotline with Seoul.

The withdrawal of workers from Kaesong and the reported deployment of missiles are being seen as an attempt to raise fears of an impending crisis in the hope of winning concessions from Washington. There are no signs that the regime is mobilising its army of 1.2 million in preparation for any sort of attack.

The chairman of the US joint chiefs of staff, General Martin Dempsey, said he did not believe the North would take military action, but added: "I can't take the chance that it won't."

The US has brought forward the deployment of a missile defence system to Guam, a US territory in the Pacific. It has drawn up plans with South Korea on how to respond to specific military provocations by the North.

The allies would "respond more forcefully than in the recent past but in a limited way intended to prevent an escalation to broader war", the New York Times said, citing Pentagon officials.

A nuclear test would anger China, which has shown signs of growing frustration with North Korea, a major recipient of Chinese aid.

The Chinese president, Xi Jinping, alluded to North Korean provocations when he said no country "should be allowed to throw a region and even the whole world into chaos for selfish gain" in a speech on Sunday.

In a telephone conversation with the UN secretary general, Ban Ki-moon, at the weekend, China's foreign minister, Wang Yi, said Beijing "opposes provocative words and actions from any party in the region and do not allow troublemaking on China's doorstep", according to the ministry's website.

The commander of the 28,000 US troops based in South Korea, General James Thurman, abruptly cancelled a visit to Washington where he had been due to testify before congressional committees. Thurman would remain in Seoul as a "prudent measure", according to a US military official quoted by Associated Press.

The chairman of South Korea's joint chiefs of staff, General Jung Seung-jo, also cancelled a visit to Washington later this week in light of rising tensions at home.

Dan Pfeiffer, an adviser to US president Barack Obama, said the administration would not be surprised if the North conducted a missile test or similar provocation.

Pfeiffer told ABC's This Week the North should "stop their actions, start meeting their international obligations and put themselves in a position where they can achieve what is their stated goal, which is economic development, which will only happen if they rejoin the international community".