Afghanistan President Ashraf Ghani said on Sunday that he will not release 5,000 Taliban prisoners ahead of peace talks next week, as laid out in a peace agreement that the U.S. signed with the Taliban on Saturday, according to AP.

Why it matters: Ghani’s public disagreement with the contents of the agreement presents the first major hurdle in its implementation, which is crucial to ending America’s longest war.

What they're saying: Ghani said in a news conference that the U.S. could not promise a prisoner swap because it is his government's sovereign right to release and accept prisoners. He said he is not ready to release prisoners before negotiations begin.

“The request has been made by the United States for the release of prisoners and it can be part of the negotiations but it cannot be a precondition,” Ghani said.

The other side: Secretary of State Mike Pompeo would not directly address Ghani's comments in an interview on CBS News' "Face the Nation" Sunday, instead saying that the U.S. "will work with all relevant parties to build on confidence, to create confidence-building measures amongst all of the parties, the Afghan government, non-Taliban, and others in the Afghan."

"No one is under any illusion that this will be straightforward," Pompeo said. "We've built an important base where we can begin to bring American soldiers home, reduce the risk of the loss of life of any American in Afghanistan, and hopefully set the conditions so the Afghan people can build out a peaceful resolution to their, now what for them is a 40-year struggle."

Go deeper: U.S. reaches "huge milestone moment" in Afghanistan peace process