Mark Wilson/Getty Images

Speaker John Boehner (R-Ohio) on Tuesday tried to explain why he has had trouble moving legislation through the House. "You might notice I have a few knuckleheads in my conference," he told the International Franchise Association.

"On any given day, 16 of my members decide they're going to go this way, and all the sudden I have nothing," he explained, referring to right-wing members of the House who have often clashed with the speaker. The GOP's majority in the lower chamber, he said, is a "paper majority."

This is not the first time that Boehner has lashed out at conservatives, who have often made life difficult for him. As recently as this summer, a conservative revolt forced the House Republican leadership to abandon a bill to address a humanitarian crisis at the border. The House eventually passed a bill that accommodated the conservatives' demands.