House Speaker Paul Ryan said Thursday he is allowing Rev. Patrick Conroy to keep his job as chaplain of the House of Representatives following outrage over Ryan's attempt to oust him from the role.

Conroy on Thursday wrote in a two-page letter to Ryan, the retiring Wisconsin Republican, that he has never "heard a complaint about my ministry" as House chaplain and was rescinding his resignation.