For years, Republican candidates have been getting crushed by Democrats in the race for black votes. As the party gears up for 2016, it’s something the GOP is eager to change—but it won’t be easy.

Republican Gov. Chris Christie recently delivered the keynote address at an annual New Jersey NAACP conference for the first time in five years. The potential presidential contender portrayed himself as a governor who has tried to seek middle ground, even as he’s butted heads with black leaders on several issues, including education, policing and the appointment of African-American judges.

His remarks – during which at least two NAACP presidents of two city chapters walked out – come as other potential 2016 Republican presidential candidates are also trying to make inroads with African-American voters, who tend to vote Democratic and twice came out in force for Barack Obama.

Last Friday, Sen. Rand Paul of Kentucky called the GOP’s failure to reach out to blacks the biggest mistake it’s made in recent decades. He did so after meeting with civil rights leaders in Ferguson, Missouri ahead of a new wave of protests following the police killing of an unarmed black teen in August. And Rep. Paul Ryan of Wisconsin has also visited a slew of urban neighborhoods in the past year-and-a-half, releasing an anti-poverty plan over the summer. Both Paul and Ryan have been discussed as potential 2016 contenders.

PHOTO ESSAY: Rising stars for 2016

The Republican National Committee has, after all, made it clear that it needs to do a better job reaching out to minorities, particularly African Americans. Since 2013, the RNC has spent millions of dollars in minority outreach efforts (even setting up an “African American Engagement Office” in Detroit). Chairman Reince Priebus has said he hopes the party will score a double-digit percentage of the black vote in 2016.

In 2012, President Obama won 93% of the black vote. The U.S. Census Bureau estimated that in that year for the first time, African-American voters cast ballots at a higher rate than whites.

But whether the GOP push actually works is yet to be seen. It’s been hampered by conservative members of the party, as well as some political analysts, who have argued that Republicans don’t need black votes to win, and instead should focus on increasing their edge among whites. These same conservative have supported strict voter identification laws and other voting restrictions, which have done nothing to improve the party’s standing with minorities. And if Democrats pick Hillary Clinton as their presidential nominee, Republicans will be up against a candidate likely to enjoy solid African-American support, as her husband did.

“They’d be lucky to reach the standard of 8%,” said Lorenzo Morris, political science professor at Howard University. Morris said the potential 2016 candidates have so far made no “concrete promises,” especially on several issues that minority voters care about, like housing regulations, unemployment and urban development. And part of the GOP’s outreach, Morris suggested, is designed to appeal not to blacks but to white moderates in the party.

At the NAACP event on Saturday, Christie – who won a sizeable 21% of the African-American vote in his re-election last year – zeroed in on laws he has backed that are supported by the civil rights group, like bail reform and a bill requiring police car cameras. But that didn’t stop Colandus Francis, the NAACP delegate from Camden County, from walking out, along with another local NAACP official. Francis pointed to the state’s decision to essentially take over the city of Camden’s troubled school district, as well as disbanding the police force and replacing it with a new county force, angering police unions.

Francis said Christie has done little to help African-Americans, noting the last time he spoke in front of the NAACP was when he was running for governor back in 2009. “Every year he has been invited, he’s never come until this year when he is obviously running for president,” said Francis. “Anyone who can’t see that is naïve and blind.”

Richard Smith, New Jersey’s NAACP president, said he believes Christie’s remarks were “sincere” and that while the governor may have 2016 on his mind, Smith – only elected last year— nonetheless wants to work with him to protect civil liberties in the state.

“Any politician, whether they be a Democrat or Republican – when they have an opportunity to be in front of a black audience and get a photo op, they take it. Politics will be politics, but I can’t worry about what the politician is doing with that photo op.”

As for Paul, Hilary Shelton, director of the NAACP Washington bureau, said the senator has reached out to the NAACP several times in the past year (Shelton said 2016 did not come up). After the midterm elections Shelton said he expects “there will be an even bigger push [from potential candidates] to talk to us and hear from us, which we appreciate.”

Meanwhile, community organizer Bob Woodson said both he and Ryan have traveled to about a dozen urban neighborhoods, including in Milwaukee, Cleveland, Indianapolis, Washington D.C., Denver and San Antonio over the past 18 months. “He wanted to expand his own understanding of what our options for people are that he’s not getting from policy makers,” said Woodson, president of the Center for Neighborhood Enterprise, and a longtime mentor to Ryan.

Woodson insisted that courting potential African American voters “was one of the consequences of the trips. But it wasn’t the purpose of it,” noting Ryan didn’t solely visit minority neighborhoods.

Some liberal commentators have expressed skepticism about the depth of Ryan’s commitment to African-American issues, noting that his draconian budget would slash services that many in the community rely on.

It may be no coincidence that Christie, Paul and Ryan have all come under fire for their remarks on civil rights and other minority issues in the past. In 2012, Christie, arguing for a referendum on same-sex marriage, said civil rights activists “would have been happy to have a referendum on civil rights rather than fighting and dying in the streets of the South”—a glaring misstatement of history, since the white majority in most southern states opposed civil rights for blacks. He later apologized. In June, the ACLU in New Jersey gave Christie a D+ rating on civil rights and liberties for his first term.

Ryan was heavily criticized, including by the Congressional Black Caucus, this spring after saying in a radio interview that there was a “tailspin of culture in our inner cities in particular, of men not working and just generations of men not even thinking about working or learning to value the culture of work.” Ryan apologized, saying his comments were “inarticulate” but insisting they had nothing to do with race.

Paul’s previous remarks on the Civil Rights Act, which banned discrimination based on race, religion, sex or national origin, have also come under fire. In a 2010 interview with the Louisville Courier-Journal, Paul praised the law for wanting to end discrimination in the public domain but said he didn’t completely approve of the government’s role, especially when it comes to private businesses. “I think it’s a bad business decision to exclude anybody from your restaurant—but at the same time, I do believe in private ownership,” he said. Paul has maintained, however, that he has never opposed the landmark legislation.

But he has been more active than his potential competitors in backing policies that help African-Americans. He has called for reducing mandatory minimum criminal sentences for nonviolent offenders and restoring voting rights for some some felons – issues that disproportionately affect minorities. And he’s said that the GOP should back off voter ID laws earlier this year, before backtracking.

UPDATE, 10/17/14: Paul told Politico, in a piece published Friday, that whoever becomes the 2016 GOP presidential candidate could rake in one-third or more of the African-American vote by focusing on education, criminal justice reform and the economy.