The prime minister's office said in a brief statement that Mahathir submitted his resignation to the palace at 1 p.m. but gave no further details.
The stunning turn of events come amid plans by Mahathir supporters to team with opposition parties to form a new government and thwart the transition of power to his named successor Anwar Ibrahim, replaying their decades-old feud.
Minutes before his resignation was offered, Mahathir's Bersatu party announced it would leave the alliance and support Mahathir as the premier. Eleven other lawmakers, including several Cabinet ministers, also announced they are quitting Anwar's party.
With some 50 lawmakers from Bersatu and Anwar's party leaving the ruling alliance, the maneuvers leave doubt whether Anwar has enough support to take power.
Mahathir and Anwar were Malaysia's top two leaders in Mahathir's first stint as premier but fell out politically before reuniting in the political pact that ousted a corruption-tainted government in the May 2018 election. Their relationship has been testy, with Mahathir refusing to set a date to relinquish power despite a preelection agreement to hand over power.
Anwar said earlier Monday that he was satisfied the government's reform agenda will continue. He refused to say more.
Ironically, the maneuvers could restore to power the Malay party of disgraced former leader Najib Razak, who with several of his party leaders are standing trial for corruption. It could also propel to national power a fundamentalist Islamic party that rules two states and champions Islamic laws. The two Malay parties still have strong support from ethnic Malays, who account for 60 percent of Malysia's 32 million people.
Anwar and his wife Wan Azizah Wan Ismail, who is currently the country's deputy prime minister, were due to meet the king on Monday. Wan Azizah tweeted that "men can plan but Allah decides," urging supporters to believe God will side with those who are patient.
Analysts said King Sultan Abdullah Sultan Ahmad Shah could decide which faction has the majority support in Parliament or call a snap election. They warned that such a new government could give rise to Malay Islamic supremacy that will derail Malaysia's multiethnic society.
"Mahathir's top political priority is to stave off Anwar's increasingly vigorous claim on the premiership. So he had to work with otherwise unsavory opposition parties to form a working parliamentary majority to counter and warn off Anwar," said Oh Ei Sun, a senior fellow at the Singapore Institute of International Affairs. "If the new government goes through, Malaysia is heading toward a a very regressive stage whereby racial supremacy and religious extremism would become the rule of the day."