Snowbirds beware: The federal government will use its planned border exit-tracking system to avoid paying hundreds of millions of dollars in social benefits now going to people who shouldn't receive them due to absences from Canada.

Newly obtained memos say the Canada Revenue Agency and Employment and Social Development Canada expect to save between about $194 million and $319 million over five years once the long-anticipated system is fully in place.

Federal officials have been working quietly to satisfy privacy commissioner Daniel Therrien's office that personal information will be properly collected, used and disclosed under the program.

Under the 2011 perimeter security pact, Canada and the United States agreed to set up co-ordinated systems to track entry and exit information from travellers.

For the moment, the tracking system involves exchanging entry information collected from people at the land border — so that data on entry to one country serves as a record of exit from the other.

The first two phases of the program have been limited to foreign nationals and permanent residents of Canada and the United States, but not citizens of either country.

The initiative was to be expanded by June 30 of last year to include information-sharing on all travellers crossing the land border.

Tracking flights as well as car trips

In addition, Canada planned to begin collecting information on people leaving by plane — something the United States already does — by requiring airlines to submit passenger manifest data for outbound international flights.

Federal officials have said work continues on the final phases, though no revised dates have been disclosed. The U.S. has legislative authority to proceed, but Canada would need to pass a bill.

A summer 2014 memo, recently released under the Access to Information Act, says savings can be expected through "preventing abuse and eligibility fraud" with respect to the employment insurance, old age security and child tax benefit programs by ensuring Canadian residency requirements are fulfilled.

It estimates savings over five years of:

$48 million by Employment and Social Development Canada for the old age security program.

$21 million by Employment and Social Development Canada for the employment insurance program.

$125 million to $250 million by the Canada Revenue Agency for the child tax benefit program.

For instance, if a Canadian citizen or permanent resident was out of Canada for more than 183 days, entry-exit information would be shared with the revenue agency to administer the child tax benefit, says an explanatory memo.

However, this information alone would "never form the basis" for action against someone, as it is merely intended as a tipsheet. Verification would be needed before a federal agency could crack down on the traveller.

Information sharing must be 'proportionate'

It has long been known that information from the entry-exit initiative would also be used to track the movement of suspected fugitives, child sex offenders, smugglers and terrorists, as well as identify people who remain in Canada past visa-expiration dates and help determine when those slated for deportation have voluntarily left.

The initiative's scope prompted the federal privacy commissioner's office to express concern it had expanded "beyond its initial parameters," says one memo. But Canada Border Services Agency officials felt the objectives were "entirely consistent" with the perimeter security pact's commitment, the memo adds.

Therrien's office is waiting to receive detailed assessments from federal agencies as to how they would use entry-exit information, said Valerie Lawton, a spokeswoman for the privacy watchdog.

"We've emphasized that each institution will need to demonstrate that the proposed collection and sharing is necessary and effective, undertaken in the least privacy-invasive manner possible and designed so that any loss of privacy is proportionate to societal benefits."

The border services agency had no immediate comment.

However, the internal notes describe several measures to protect privacy including signs at the border to notify travellers their information may be used for program integrity. People flying into Canada have been warned for many years that information on their customs declaration card may be shared.

Legislation to implement the final phases of the entry-exit initiative will spell out exactly how the information may be used and disclosed, and there will be redress procedures under which people can request access to their personal information, ask for corrections if needed and file complaints.

Beware the IRS, too?

Cross-border tax adviser Matt Altro told CBC News the new measures are "quite upsetting to hear" and his clients won't be impressed.

"If you're going to maintain your Canadian tax residency, and continue to pay tax in Canada, you should be able to benefit from these programs [like OAS]," said Altro, the CEO of MCA Cross Border Advisors.

He said it doesn't make much sense to crack down on snowbirds, unless the real goal is encouraging retirees to give up their Canadian residency status — which can have tax advantages, like a lower tax rate on RRSP withdrawals.

"If you become a non-resident of Canada, you are not subject anymore to OAS clawback," he said. "You benefit from it better than a Canadian would."

However, changing residency also affects a retiree's ability to access the Canadian health-care system.

"Canadian snowbirds need to be aware that the [U.S. Internal Revenue Service] is probably already ... following suit, and could bring up a whole host of issues on the U.S. side if you're spending too much time in the U.S.," he said. "You need to be working with a cross-border [tax] professional who knows how the system works."

The crackdown could affect not only retirees but younger Canadians who spend large stretches of time abroad.