WASHINGTON, United States (AFP) — Paul Ryan may be in his final term as speaker of the US House of Representatives and could leave Congress by year end, Politico reported Thursday, in a report that could set off a scramble for a successor.

The news outlet cited several people who know Ryan, including fellow lawmakers, congressional aides, conservative intellectuals and party lobbyists, saying they did not expect him to remain in Congress beyond 2018.

Asked directly after a Thursday press conference whether he would be stepping down soon, Ryan said, “I’m not… no.”

A spokeswoman for Ryan’s office, AshLee Strong, called the report “pure speculation.”

“As the speaker himself said today, he’s not going anywhere any time soon,” she added.

But the report appeared to heighten the conjecture about the Wisconsin lawmaker’s future.

Ryan, 47, made no secret about his hesitation in taking the top congressional job in 2015, after his predecessor John Boehner abruptly announced he was retiring when he faced a revolt from right-wing conservatives.

He also spoke out critically against Donald Trump during the presidential race.

But Ryan has developed a better-than-expected relationship with Trump, and has worked with him on several landmark issues including health care and the current big legislative push, tax reform. The White House made it clear Trump wants Ryan to stay.

“The president did speak to the speaker not too long ago, and made sure that the speaker knew very clearly, in no uncertain terms, that if the news was true he was very unhappy about it,” White House spokeswoman Sarah Sanders said.

It is not known whether Ryan would run for his congressional seat in next November’s mid-term elections. But should he announce his departure well ahead of time, it would dramatically diminish his deal-making leverage and his ability to raise money for the party.

“Ryan’s preference has become clear: He would like to serve through Election Day 2018 and retire ahead of the next Congress,” Politico reported.

Some Republicans in Congress were already anticipating a leadership battle.

“Brace yourselves for the mother of all barn cleanings,” tweeted House conservative Thomas Massie, who has been a thorn in the Republican leadership’s side.