PALM BEACH, Fla. — President Donald Trump and Japanese Prime Minister Shinzo Abe said Wednesday they had failed to reach a deal that would exempt Japan from new U.S. steel and aluminum tariffs, as Abe had wanted.

Instead, the leaders announced they had agreed to start talks on a new “free, fair and reciprocal” trade agreement between the two countries following two days of talks.

“If we can come to an arrangement on a new deal, that would certainly be something we would discuss,” Trump said when asked about the tariffs during a joint press conference at his private Mar-a-Lago club.

But he said the current trade deficit between the two countries was too high to merit an exemption now. According to the U.S. Department of Commerce, the trade deficit was $56.1 billion last year.

It was a disappointment for Japan. Most other key U.S. allies — among them Australia, Canada, the European Union and Mexico — have already been granted exemptions to Trump’s protectionist measures. And Japan has previously voiced reluctance to negotiating a bilateral trade deal with the U.S., saying it would prefer the U.S. rejoin the Trans-Pacific Partnership.

But Abe, whose visit comes as Trump prepares for a historic summit with North Korean leader Kim Jong Un, appeared to win assurances from Trump that he and the president were on the same page when it comes to the Kim meeting.

Perhaps the most significant for Abe: Trump’s promise to raise with Kim the issue of Japanese citizens who have been abducted by the North.

Trump said it was clear from their discussions that the abductees were “one of the truly most important things on Shinzo’s mind.” Trump said he wanted to see the “families reunited as soon as possible.”

Pyongyang has acknowledging abducting 13 Japanese, while Tokyo maintains North Korea abducted 17. Five have been returned to Japan. North Korea says eight others have died and denies the remaining four entered its territory. Japan has demanded further investigation, and Abe has made the issue one of his top priorities.

The U.S. has also been pushing for the release of three Americans held captive. CIA Director Mike Pompeo raised their plight during his meeting earlier this month with Kim, according to a senior White House official who was not authorized to disclose the sensitive discussions and spoke to The Associated Press on condition of anonymity.

Trump expressed optimism Wednesday, saying that officials had been working “very diligently” to secure the Americans’ return and that there was a “good chance” it would happen.

But on trade, Trump made clear he would prefer a bilateral trade deal with Japan instead of rejoining the 11-nation TPP Pacific Rim trade pact.

“I don’t want to go back into TPP, but if they offered us a deal that I can’t refuse on behalf of the United States, I would do it,” Trump said. Still, he added, “I like bilateral better. I think it’s better for our country, I think it’s better for our workers, and I much would prefer a bilateral deal, a deal directly with Japan.”

Trump and Abe also appeared to agree to disagree on the issue of steel and aluminum imports. Abe argued during the press conference that Japanese imports “would not exert any negative influence” on the security of the United States and said the high quality of Japan’s steel and aluminum products would be difficult to replace.

The visit included a mix of business and pleasure. Trump and Abe spent Wednesday morning golfing at one of the president’s nearby golf courses. On Tuesday evening, the president and first lady hosted Abe and his wife at an intimate al fresco dinner on Mar-a-Lago’s patio. And the group reconvened for a second, larger dinner Wednesday evening.

Despite their disagreements, officials from both countries insist the personal relationship that Trump and Abe share has never been stronger. The leaders have bonded over “golf diplomacy” in both the U.S. and Japan, and U.S. officials note that Trump has met with Abe more than any other foreign leader.

During the trip, Abe praised Trump for his courage in agreeing to meet with Kim and said that Kim’s willingness to meet was a sign that the pressure campaign against the North was working.

But he also urged caution, saying there should be “no reward” given to the North just because the reclusive country is responding to dialogue. He said that “maximum pressure should be maintained.”

On Tuesday night, media outlets including The Associated Press reported that Pompeo recently traveled in secret to North Korea to meet with Kim ahead of a U.S.-North Korea summit planned in the next two months. Trump confirmed the meeting in a tweet Wednesday, saying “a good relationship was formed” heading into his anticipated summit with Kim.

“Denuclearization will be a great thing for World, but also for North Korea!” Trump wrote.

Associated Press writers Ken Thomas, Darlene Superville and Matthew Pennington in Washington contributed to this report.