Prime Minister Stephen Harper left for South America Tuesday for four days of bilateral talks and trade meetings, after addressing his caucus over the growing Senate expense controversy.

Harper's visit to Peru and Colombia comes at a time when the Prime Minister's Office is under increasing scrutiny over the decision by Nigel Wright, Harper's chief of staff, to give Senator Mike Duffy $90,000 so the former journalist could repay his ineligible expenses.

Wright announced on Sunday that he was resigning his post due to the controversy. On Monday, the NDP called for the RCMP to investigate the actions of Wright.

Harper told the Conservative MPs and senators gathered in a meeting room on Parliament Hill he was "upset" by the conduct of some parliamentarians and even his own office, but did not mention any names and offered no further details on his former chief of staff's actions.

"Anyone who wants to use public office for their own benefit should make other plans, or better yet, leave this room," Harper said, before journalists and cameras invited to hear his remarks were asked to leave the room. He did not take questions.

The regularly scheduled meeting of the Conservative Party caucus was moved up a day to Tuesday to accommodate the prime minister's travel plans. Harper is likely to hear more questions about the Senate expenses controversy during his trip, but his official focus will be on trade and bilateral relations.

First bilateral visit to Peru

Harper will begin his visit to South America by meeting with the president of Peru, Ollanta Humala Tasso, in Lima, on May 22.

It will be Harper's second time in the country, and the first bilateral visit by a sitting Canadian prime minister to Peru. Harper's first visit was during the Asia-Pacific Economic Co-operation summit in 2008.

"Canada is committed to increasing trade with the Americas to create economic opportunities for Canadians and people throughout the hemisphere," the prime minister said in a written statement, adding that he is looking forward "to working with our partners in the Americas to achieve these goals, and address shared security and development challenges."

Harper will be accompanied by Minister of State of Foreign Affairs (Americas and Consular Affairs) Diane Ablonczy. The two will be expected to meet with members of the Peruvian cabinet and Canadian investors who are doing business in Peru.

The focus of the visit, according to Harper's director of communications, Andrew MacDougall, will be on "deepening the bilateral relationship with Peru which includes partnerships for education, economic prosperity and defence with an emphasis on working together in the extractive sector."

Trade summit in Colombia

The prime minister will travel from Peru to Cali, Colombia, where he will attend the Pacific Alliance Leaders' Summit.

This year will mark the 60th anniversary of full diplomatic relations between Canada and Colombia, with two-way merchandise trade totalling nearly $1.5 billion in 2012.

Over 70 Canadian companies have foreign direct investment in the oil and gas, mining, financial, and agri-food sectors in Colombia.

While the two countries enjoy a solid trade relationship, Colombia still faces many challenges in way of combating drug trafficking and transnational crime.

This will be the first time Harper will attend a meeting of the Pacific Alliance since Canada gained observer status in 2012. Other observer countries include Australia, Costa Rica, Guatemala, Japan, New Zealand, Panama, Spain and Uruguay.

The purpose of the alliance, said MacDougall, "is to facilitate the free movement of goods services, capital and people among member countries and to strengthen trade and investment ties with Asia."

Launched in April 2011, the Pacific Alliance is a group of four countries with Latin America's fastest-growing economies: Chile, Colombia, Mexico and Peru.

Canada has bilateral free trade agreements with all four, and two-way trade with the Pacific Alliance countries reached $39 billion in 2012.

Full membership in the Pacific Alliance, would give Canada a "second NAFTA", according to Carlo Dade, an independent trade analyst and former executive director of the Canadian Foundation for the Americas.

The cost of membership would be "minimal" and "cutting tariffs with Pacific Alliance countries will be easier on Canadian producers than with the European Union or the Trans-Pacific Partnership," said Dade.

While Canada may very well be eyeing membership in the Pacific Alliance, MacDougall said the group of four is in its early stages.