It's just a matter of time. There will be a backlash against Rand Paul's efforts at minority outreach, spearheaded by Republicans who see this as an opening to the Kentucky senator's right in the 2016 presidential primaries, especially in the wake of the crisis in Ferguson.

Paul's outreach to African-Americans has largely consisted of rhetoric very much in line with his libertarian principles, such as criticizing the prison industrial complex and the militarization of local police. And right now, the discontent on the right is mostly rumbling beneath the surface, on conservative listservs and less prominent special-interest websites. But already, Michael Gerson fired a public warning shot at Paul in The Washington Post.

If Jack Kemp's project of expanding the Republican Party's appeal to African-Americans and other minority voters truly does fall to Paul, Gerson warned, it "would be an utter, counterproductive failure," essentially because libertarianism has a tricky history on race, with Paul notably getting tangled up on whether he would have supported the Civil Rights Act in 1964.

Expect to see more of this opposition, primarily from two sources.

One group will be traditional law-and-order conservatives who disagree with the substantive positions Paul is taking to broaden the party's appeal: sentencing reform, voting rights for nonviolent felons, and criticism of police militarization and drug laws.

While there is some evidence conservatives are moving in Paul's direction on some of these issues — a New York Times/CBS News poll found 59 percent of conservatives agreed local police forces shouldn't have access to tanks and military-style weapons — the get-tough approach retains considerable appeal. The same poll found that more than two thirds expressed confidence in the investigation into the Michael Brown shooting. While 66 percent believe the Ferguson protests have gone too far, only 20 percent say the same about the police response.

Indeed, another 20 percent of conservatives told the New York Times/CBS News pollsters that the police response hadn't gone far enough. Thirty-eight percent said it was just right. Inevitably, some Republicans will look at these numbers and charge Paul with being soft on crime.

The second group of Paul critics will be made up of Republicans who disagree with Paul on other issues, such as his strong fiscal conservatism or his departure from George W. Bush's aggressive foreign policy. They will preview liberal attacks on Paul as a hypocrite on race in an effort to derail his candidacy. Put Gerson firmly in this camp.

Gerson brought up Paul's MSNBC remarks on the Civil Rights Act, essentially taking the Rachel Maddow line that they amounted to an assault on the landmark legislation. He added that Paul "worked for a presidential candidate in 2012 (his father, Ron Paul) who claimed that the Civil Rights Act 'violated the Constitution and reduced individual liberty'."

The former Bush speechwriter turned Post scribe also dinged Paul over a controversial former aide "who authored a column titled 'John Wilkes Booth Was Right.'" "This personnel decision would have been impossible to imagine from Kemp," Gerson said. (The aide in question, Jack Hunter, penned a response to Gerson.)

Any intellectually honest person must admit that Paul's platform and the issues that attracted large crowds of young people to his father's campaigns have little to do with any of this baggage. The youthful activists behind both Pauls are more likely to care about how police brutality impacts minorities than the racial outrage du jour on talk radio.

Paul is challenging his party's longstanding positions on crime at some political risk. Whatever else can be said of this approach, it is the polar opposite of the Lee Atwater libertarianism embodied by his father's newsletters.

The younger Paul should be respectful of his party's old law-and-order consensus. It originated at a time when crime was genuinely spiraling out of control. The subsequent drop in crime that makes Paul's libertarian pitch possible owes something to conservative tough-on-crime policies —the pendulum, having swung too far toward leniency in the 1960s, has now merely swung too far in the opposite direction.

A genuine limited-government political movement must concern itself with military-style law enforcement in small towns as much the capital gains tax rate. That is a matter of philosophical integrity as well as political outreach.

It would be a shame if conservative progress in this area is set back because neither Rand Paul's movement nor the rest of the Republican Party are far enough removed from the controversies of 50 years ago.