One Sunday morning last December, China’s defense ministry summoned military attachés from several embassies to its monolithic Beijing headquarters.

To the foreigners’ surprise, the Chinese said that one of their nuclear-powered submarines would soon pass through the Strait of Malacca, a passage between Malaysia and Indonesia that carries much of world trade, say people briefed on the meeting.

Two days later, a Chinese attack sub—a so-called hunter-killer, designed to seek out and destroy enemy vessels—slipped through the strait above water and disappeared. It resurfaced near Sri Lanka and then in the Persian Gulf, say people familiar with its movements, before returning through the strait in February—the first known voyage of a Chinese sub to the Indian Ocean.

The message was clear: China had fulfilled its four-decade quest to join the elite club of countries with nuclear subs that can ply the high seas. The defense ministry summoned attachés again to disclose another Chinese deployment to the Indian Ocean in September—this time a diesel-powered sub, which stopped off in Sri Lanka.

China’s increasingly potent and active sub force represents the rising power’s most significant military challenge yet for the region. Its expanding undersea fleet not only bolsters China’s nuclear arsenal but also enhances the country’s capacity to enforce its territorial claims and thwart U.S. intervention.

China is expected to pass another milestone this year when it sets a different type of sub to sea—a “boomer,” carrying fully armed nuclear missiles for the first time—says the U.S. Office of Naval Intelligence, or ONI.

China is hardly hiding its new boomers. Tourists could clearly see three of them at a base opposite a resort recently in China’s Hainan province. On the beach, rented Jet Skis were accompanied by guides to make sure riders didn’t stray too close.

These boomers’ missiles have the range to hit Hawaii and Alaska from East Asia and the continental U.S. from the mid-Pacific, the ONI says.

“This is a trump card that makes our motherland proud and our adversaries terrified,” China’s navy chief, Adm. Wu Shengli, wrote of the country’s missile-sub fleet in a Communist Party magazine in December. “It is a strategic force symbolizing great-power status and supporting national security.”

To naval commanders from other countries, the Chinese nuclear sub’s nonstop Indian Ocean voyage was especially striking, proving that it has the endurance to reach the U.S. Pacific Fleet’s headquarters in Hawaii.

“They were very clear with respect to messaging,” says Vice Adm. Robert Thomas, a former submariner who commands the U.S. Seventh Fleet, “to say that, ‘We’re a professional navy, we’re a professional submarine force, and we’re global. We’re no longer just a coastal-water submarine force.’ ”

In recent years, public attention has focused on China’s expanding military arsenal, including its first aircraft carrier and stealth fighter. But subs are more strategically potent weapons: A single one can project power far from China and deter other countries simply by its presence.

China’s nuclear attack subs, in particular, are integral to what Washington sees as an emerging strategy to prevent the U.S. from intervening in a conflict over Taiwan, or with Japan and the Philippines—both U.S. allies locked in territorial disputes with Beijing.

And even a few functional Chinese boomers compel the U.S. to plan for a theoretical Chinese nuclear-missile strike from the sea. China’s boomer patrols will make it one of only three countries—alongside the U.S. and Russia—that can launch atomic weapons from sea, air and land.

“I think they’ve watched the U.S. submarine force and its ability to operate globally for many, many years—and the potential influence that can have in various places around the globe,” says Adm. Thomas, “and they’ve decided to go after that model.”

China's nuclear-sub deployments, some naval experts say, may become the opening gambits of an undersea contest in Asia that echoes the cat-and-mouse game between U.S. and Soviet subs during the Cold War—a history popularized by Tom Clancy's 1984 novel "The Hunt for Red October."

Back then, each side sent boomers to lurk at sea, ready to fire missiles at the other’s territory. Each dispatched nuclear hunter-killers to track the other’s boomers and be ready to destroy them.

The collapse of the Soviet Union ended that tournament. But today, as China increases its undersea firepower, the U.S. and its allies are boosting their submarine and anti-sub forces in Asia to counter it.

Neither China nor the U.S. wants a Cold War rerun. Their economies are too interdependent, and today’s market-minded China doesn’t seek global revolution or military parity with the U.S.

Chinese officials say their subs don’t threaten other countries and are part of a program to protect China’s territory and expanding global interests. Chinese defense officials told foreign attachés that the subs entering the Indian Ocean would assist antipiracy patrols off Somalia, say people briefed on the meetings.

Asked about those meetings, China’s defense ministry said its navy’s activities in the Indian and Pacific Oceans “comply with international law and practice, and we maintain good communication with all relevant parties.”

Submarines help Beijing fulfill international duties without changing its defense policy, says China’s navy spokesman, Sr. Capt. Liang Yang. “If a soldier originally has a handgun, and you give him an assault rifle, you’ve increased his firepower, but his responsibilities haven’t changed.” He declines to comment on boomer patrols.

Still, the U.S. has moved subs to the forefront of its so-called rebalancing, a strategy of focusing more military and diplomatic resources on Asia. Sixty percent of the U.S. undersea force is in the Pacific, U.S. naval commanders say, compared with half the U.S. surface fleet. The U.S. Navy plans to station a fourth nuclear attack sub in Guam next year, they say.

Since December, the U.S. has positioned six new P-8 anti-submarine aircraft in Okinawa, Japan. The U.S. has also revitalized an undersea microphone system designed to track Soviet subs and is testing new technologies such as underwater drones to search for Chinese subs.

Related Article: As China Deploys Nuclear Submarines, U.S. P-8 Poseidon Jets Snoop on Them

Several nearby countries, including Australia, have said they plan to expand or upgrade their submarine and anti-sub forces. Vietnam, which is embroiled in a territorial dispute with China, has since December received at least two of the six Russian-made attack subs it has ordered.

Australia’s navy chief, Vice Admiral Tim Barrett, told a parliamentary committee on Wednesday that the 12 subs his country is buying to replace its six-strong current fleet would need to operate far afield, potentially in contested areas of the South China Sea. “There are other nations in the area that are building their submarine forces as well,” he said. “The issue for us is to be able to consider that we may need to counter those things.”

Rear Adm. Phillip Sawyer, the commander of U.S. submarine forces in the Pacific, says that many more submarines are now operating in the region than during the Cold War. “One of my biggest concerns truthfully is submarine safety,” he says on a recent dive aboard the USS Houston, a nuclear-attack sub based in Hawaii. “The more submarines you put in the same body of water, the higher the probability that they might collide.”

China now has one of the world’s biggest attack-sub fleets, with five nuclear models and at least 50 diesel models. It has four boomers, the ONI says.

Beijing’s quest for a nuclear-sub fleet dates to the 1960s, say Chinese historians. Mao Zedong once declared, “We will build a nuclear submarine even if it takes us 10,000 years!”

China has used diesel subs since the 1950s, but they have proved easy to find because they must surface every few hours. Nuclear subs are faster and can stay submerged for months. China launched its first nuclear sub on Mao’s birthday in 1970 and test-fired its first missile from underwater in 1988, although its first boomer never patrolled carrying armed nuclear missiles, U.S. naval officers say.

Adm. Liu Huaqing, the founder of China's modern navy, outlined the role of nuclear attack subs in his overall strategy in the 1980s, Chinese historians say. He saw China as constrained by U.S. forces aligned in both a "First Island Chain" stretching from southern Japan to the Philippines and a "Second Island Chain" from northern Japan via Guam to Indonesia. He argued that China should establish naval dominance within the first chain by 2010, within the second chain by 2020 and become a global naval power by 2050.

China officially unveiled its nuclear undersea forces in October 2013 in an unprecedented open day for domestic media at a nuclear-sub base. Its capabilities aren’t close to those of the U.S., which has 14 boomers and 55 nuclear attack subs.

The U.S. concern is how to maintain that edge in Asia when the Navy projects that fiscal constraints will shrink its attack-sub fleet to 41 by 2028.

Beijing isn’t likely to try matching the U.S. sub force, having studied the way the Cold War arms race drained the Soviet Union’s finances. “We’re not that stupid,” says retired Maj. Gen. Xu Guangyu, a former vice president of the People’s Liberation Army Defense Institute.

“But we need enough nuclear submarines to be a credible force—to have some bargaining chips,” he says. “They must go out to the Pacific Ocean and the rest of the world.”

China's hunter-killers pose the immediate challenge to the U.S. and its partners. Adm. Sawyer has tracked them for more than a decade, first as a commander of U.S. subs in Japan and Guam and now from his headquarters in Pearl Harbor.

On his desk is a glass-encased naval chart with white labels marking China’s submarine bases. Drawn on the map are two lines marking “First Island Chain” and “Second Island Chain.”

Over the past few years, Chinese attack subs have broken beyond the first chain to operate regularly in the Philippine Sea and have started patrolling year-round, Adm. Sawyer says. Penetrating the second chain is the next logical step, he adds: “They are not just building more units and more assets, but they’re actually working to get proficient with them and understand how they’d operate in a far-away-from-home environment.” Related article: When Sub Goes Silent, Who Has Control of Its Nuclear Warheads?

Adm. Sawyer declines to say whether China has sent a sub as far as Hawaii but says the December Indian Ocean expedition shows that it has “the capability and the endurance” to do so.

That was a Shang-class sub, a type naval experts say China first launched in 2002 that can carry torpedoes and cruise missiles. In peacetime, China would probably use these hunter-killers to protect sea lanes, track foreign vessels and gather intelligence, naval experts say. But in a conflict, they would likely try to break through the First Island Chain to threaten approaching vessels and disrupt supply lines.

Still, the two recent sub voyages highlighted a weak point for China. Its subs must use narrow straits to reach the Pacific or Indian Oceans. Those chokepoints—among them, the Malacca, Sunda, Lombok, Luzon and Miyako Straits—can be relatively easily monitored or blockaded.

Moreover, China’s anti-sub capabilities remain relatively weak. U.S. subs can track their Chinese counterparts even near China’s shores, where U.S. ships and planes are vulnerable to Chinese aircraft and missiles, American naval officers say.

Adm. Sawyer declines to say whether the U.S. tracked the Shang or how close U.S. subs get to China, saying only: “I’m comfortable with the U.S. submarine force’s capability to execute whatever tasking we’re given.”

The USS Houston returned recently from a seven-month deployment to the Western Pacific. Its commanding officer, Cmdr. Dearcy P. Davis, declines to say exactly where the sub went but adds, “I can say that we went untracked by anyone. We have the ability to break down the door if someone [else] can’t. That’s not trivial.”

China’s missile-carrying boomers present a longer-term challenge.

From the Lan Sanya beach resort in Hainan, guests can easily make out the matte-black hulls of what naval experts say are three of China’s new boomers, known as the Jin-class, and one Shang-class attack sub. As he threw open a hotel room’s curtains, a bellboy beamed with pride and pointed out the vessels across the bay. “Better not go that way,” joked a Jet Ski guide on a recent ride. “They might shoot at us.”

China hasn’t said when it might launch boomer patrols. But Western naval officers saw the October nuclear-sub event as a signal that the Jin subs and their JL-2 missiles were ready to start.

Adm. Jonathan Greenert, a former submariner who is now the U.S. chief of naval operations, says that the U.S. is waiting to see how China will use its new boomers. “Is it an occasional patrol they’re going to choose to do? Is it going to be a continuous patrol? Are they going to try to be sure that this patrol is totally undetected?” he says. “I think that’s all going to be in the equation as to our response.”

Soviet boomers ventured far into the Pacific and Atlantic into the 1970s because their missiles couldn’t reach the U.S. from Soviet waters. As missile ranges increased, Soviet subs retreated to so-called bastions, such as the Sea of Okhotsk. The U.S. deployed hunter-killers around those bastions.

Similar dynamics are at play as China decides whether to send its own boomers into the Pacific. Their JL-2 missiles can travel about 4,600 miles—possibly enough to strike the U.S. West Coast from East Asia, the ONI says. To strike more U.S. targets, they would need to lurk throughout the Pacific.

But China’s boomers probably couldn’t pass undetected through many straits, say U.S. officers and Chinese experts. “The Jin class is too noisy: It’s probably at the level of the Soviets between 1970 and 1980,” says Wu Riqiang, a former missile specialist who studies nuclear strategy at Beijing’s Renmin University. “As long as you are noisy, you won’t even go through the chokepoints.”

Early in the Cold War, the U.S. built a network of seabed microphones to listen at chokepoints leading to the Pacific and Atlantic. In recent years, the U.S. has revitalized parts of that network, called the Sound Surveillance System, or Sosus. The U.S. is also now adding mobile networks of sensors—some on underwater drones—and seeking surveillance data from Asian countries. Related Article: Underwater Drones Join Microphones to Listen for Chinese Subs

Meanwhile, China is trying to replicate Sosus, say several naval experts. A government-backed scientific journal reported last year that China had built a fiber-optic acoustic network in the South China Sea.

Over the short term, Prof. Wu says, China will probably keep its boomers near its coast, possibly in the South China Sea, which is deepest and furthest from U.S. bases. That, say some naval officers, may explain why China keeps its Jin-class subs in Hainan and why it is pressing territorial claims and hindering U.S. surveillance there.

Last November, China declared an "air-defense identification zone" over the East China Sea and warned of measures against aircraft that entered without identifying themselves in advance. Many U.S. officials expect China to do the same over the South China Sea, although Chinese officials say they have no immediate plans for that.

In August, the Pentagon said a Chinese fighter had flown dangerously close to a U.S. P-8 near Hainan. China’s defense ministry publicly said that its pilot flew safely and asked the U.S. to cease such operations.

The problem with confining boomers to the South China Sea is that Beijing fears that missiles fired from there could be neutralized by the next stages of a U.S. regional missile-defense system, Chinese nuclear experts say.

Prof. Wu, who has taken part in nuclear-strategy negotiations with the U.S., predicts that over the next two decades, China will make quieter boomers that can patrol the open sea even as the U.S. pursues a global missile-defense system.

“I hope the U.S. and China can break this cycle,” he says, “but I’m not optimistic.”

—Rob Taylor in Canberra contributed to this article.