NORTH SAANICH, British Columbia (Reuters) - Towns on the southern end of Vancouver Island on Canada’s west coast are abuzz with interest over the possibility that Britain’s Prince Harry and his wife, Meghan, will become semi-permanent residents there.

The Duke and Duchess of Sussex sparked an uproar on both sides of the Atlantic Ocean when they said on Wednesday that they would be stepping back from their senior roles in the British royal family.

The couple’s stated intention to spend more time in North America has led onlookers to suspect they will make a home in Canada, which recognizes Harry’s grandmother Queen Elizabeth as its head of state.

On Friday, Reuters reported that after just a few days back in the UK, Meghan had returned to Canada to reunite with the couple’s 8-month-old son, Archie.

According to media reports, she has gone back to the Saanich Peninsula, a part of Vancouver Island just north of British Columbia’s capital, Victoria, an area where the family took a six-week vacation at the end of last year.

“I wouldn’t call myself a (royal) follower until they come into town, and then of course it piques our interest,” Joan Wilson, a resident of Sidney, on the Saanich Peninsula, said.

Towns on the peninsula, which has long been popular with retirees thanks to its laid-back pace of life and proximity to ferries to the British Columbia mainland, mostly have fewer than 20,000 residents and are filled with ocean-themed gift shops, boutiques and fish-and-chip huts.

Slideshow ( 12 images )

The couple have captured residents’ fascination since their vacation in the area, which has “a really interesting mix of very wealthy people and middle-class people,” said North Saanich resident Beth Doman. “There’s a lot of hidden mansions, so we were all trying to figure out where they were staying.”

Despite the curiosity, locals are determined to give them privacy.

“The last thing they want is mobs of people coming around and taking their picture,” said Ron Buschgens, owner of Lolly Gobble Sweet Shop in Sidney. “That would start to settle down as people got used to the idea of them being here.”

Many residents declined to speak to reporters on the record, saying they wished to respect the couple’s privacy.

“Harry’s mother died because of all the attention that was on her and it was very tragic,” Sidney resident Lynn Salituro said, referring to the late Princess Diana. “I’m sure they just want to break away and have a safe haven and somewhere they can live a normal life.”

Queen Elizabeth has summoned Prince Harry, as well as his father, Prince Charles, and elder brother Prince William, for a crisis meeting on Monday at her Sandringham estate to discuss future arrangements, a Buckingham Palace source told Reuters.