NEW YORK (CBSNewYork) — The Rev. Al Sharpton says he will not be attending the funeral of NYPD Officer Randolph Holder.

In a letter to Holder’s father, Sharpton says he has “withdrawn my acceptance of your invitation to attend or have words” at Holder’s funeral, set for 3 p.m. Wednesday. CBSNewYork.com will have a live stream of the funeral.

Sharpton said he was invited to speak at the service by Holder’s father and his pastor as a way to “give unity to the city,” but said he is now declining “after reviewing some of the media reports this morning.”

“I thought my coming might give a sense of unity in the city; that we can disagree on cases and on policies but that we are united that the senseless and ruthless killing of officers like your son must be denounced and we must as a city come together and mourn that loss,” Sharpton said in the letter. “It is clear though after reading several articles that some union leaders and some others want to turn your and your pastor’s noble efforts into some kind of confrontation or sideshow and not keep focused on the brutal, senseless murder of your son.”

Holder died last Tuesday after being shot in the head by a fleeing suspect, police said. On Saturday, Sharpton held a prayer vigil at the growing makeshift memorial in the Harlem neighborhood where Holder served.

Sharpton didn’t mention the specific media reports he references in the letter, but Holder’s fiancee, Mary Muhammaed, told The New York Post that Holder “detested” Sharpton and would never have wanted him at the funeral.

In addition to the letter, Sharpton released a copy of the funeral program he said was sent to him Monday afternoon by Holder’s family that lists Sharpton as delivering the eulogy.

Last week, Holder’s sister, Sherry Holder, reportedly asked loudly in court why Sharpton wasn’t there during the arraignment of her brother’s suspected killer.

Sharpton responded by saying he made an attempt to visit the family after the shooting. He added he was glad Holder’s sister asked where he was, since he hoped it would facilitate a meeting with the family.

In the past, Sharpton has attended the funerals of Michael Brown in Ferguson, Mo., and Eric Garner in Staten Island, among others, and has called for national reform on police conduct.

“When I visited you on Saturday at your invitation and it was raised by you and Rev. Mullings about my being invited and having words to give unity to the city, I stated to you then that I am unapologetically committed to fighting for civil rights causes including cases that involved alleged police misconduct and brutality,” Sharpton said in the letter to Holder’s father.

But he said he has never been anti-police said he “worked alongside with police around gun violence issues.”

“I refuse, despite my strong feelings on police issues, to be part of anything that would marginalize and take away from the focus of this city and nation mourning your son tomorrow,” he said.

Sharpton said he and his National Action Network would be also be contributing $5,000 to Holder’s family to help pay for funeral costs.

Holder’s wake is being held Tuesday at the Greater Allen A.M.E. Cathedral of New York. His funeral will be held Wednesday. He will be buried in his native Guyana.