(CNN) The rumors of House Speaker Paul Ryan's looming demise are exaggerated -- at least according to the very members who would be responsible for throwing him out earlier than his planned January departure. "This 'Throw Paul Ryan overboard,' it doesn't make any sense to me," said Republican Study Committee Chairman Mark Walker, a conservative North Carolina Republican. "He's the best spokesperson we have," said Rep. Ryan Costello, a Republican from Pennsylvania who is also retiring at the end of this Congress. If Ryan leaves, the problems of the conference aren't instantly resolved, they only become the next leader's battles. "At this point, that's something for Speaker Ryan and members of Congress to make that determination, not something that the White House has weighed into at this point," Sanders told reporters.