He is the primary heir to the ultimate political dynasty. His great-grandfather, Jawaharlal Nehru, was the first and longest-serving prime minister of India. His grandmother, Indira Gandhi, was the first female prime minister of the country, and his father was India's youngest prime minister.
If the 2014 election was Congress' worst political showing ever, Thursday's poll delivered a double blow to Mr Gandhi. Congress won just over 50 seats against the 300 plus that Mr Modi's BJP got; and if that was not bad enough, he lost his own seat in the family bastion of Amethi in Uttar Pradesh.
He will still sit in the parliament though because this time he contested from a second seat - Wayanad in Kerala - which he won.
However, Amethi was a prestige battle. It was the seat from where both his parents - Sonia and Rajiv Gandhi - had contested and won and he himself had held it for the past 15 years. Even an emotional letter delivered to every house in Amethi - addressed to "Mera Amethi Parivar" - was not enough to spare him a ballot box humiliation at the hands of the BJP's Smriti Irani, a high profile actress-turned-politician.
It sits in the heart of Uttar Pradesh - India's most populous state - which is considered the ground zero of politics. It's generally believed that whoever wins the state rules the country.
Eight of 14 Indian prime ministers - including Mr Gandhi's great grandfather, grandmother and father - were from the state, which elects the largest number of MPs - 80 out of a 545-member lower house. Even Narendra Modi, who is originally from Gujarat, chose Uttar Pradesh to make his debut as an MP in 2014 when he contested from the ancient city of Varanasi.
Not many were expecting an outright win for the Congress, but they were definitely expected to do better than 2014. That's why Thursday's results have stunned many inside and outside the party. Congress may limp on in parliament, but the question many people are asking is if this means the Gandhi era is over - or whether it should be ended to revive the party's fortunes.
What does Congress want?
On Thursday evening, Mr Gandhi addressed a press conference in Delhi where he conceded the election to Mr Modi, saying the people had given their mandate and chosen the BJP. He also took full responsibility for the Congress party's defeat.
And even though counting was not over in Amethi with more than 300,000 votes yet to be counted, he conceded the constituency to Smriti Irani.
"I want to congratulate her. She has won, it's a democracy and I respect the decision of people," he said.
Refusing to give further details about the Congress performance or what would come next, Mr Gandhi said it would all be discussed in the meeting of the Congress Working Committee, the party's top decision-making body.
He also told the Congress workers, the ones who lost and the ones who won, not to lose hope. "There is no need to be afraid. We will continue to work hard and we will eventually win."
But at the Congress office in Lucknow, the future victory that Mr Gandhi promised seemed like a distant dream to the handful of despondent party workers, glued to a TV screen, watching the bloodbath unfold as several party veterans lost their seats.
"Our credibility is very low. People have no faith in our promises. They are not trusting what we are saying," one party official who didn't want to be named told me.
"Mr Modi failed to fulfil the commitments he made, but people still believe him."
I ask him why?
"Even we can't understand why!" he says.
The dismal performance of the Congress is bound to raise questions over Mr Gandhi's leadership and many analysts are already calling for a change, demanding that he step down from the top party post. But all such calls, like in the past, have come from outside the party and are likely to be rejected by its leadership.
As rumours swirled around Delhi that Mr Gandhi had offered to quit, Congress politician Mani Shankar Aiyar told BBC Hindi that "Congress will not question its leadership and [will] not accept Mr Gandhi's resignation were he to offer it".
He added that the leadership was not the reason for the party's resounding defeat. "It's the other reasons we need to work on," he said.
A local party spokesman in Lucknow, Brijendra Kumar Singh, explained that, in their view, the problem was not with Gandhi power, but with party infighting and poor campaign choices.
"There are weaknesses in the party structure, there's infighting within the ranks, we were late getting off the ground with our campaign, and our attempts - though unsuccessful - to join the alliance of regional parties in Uttar Pradesh and Bihar were a bad idea."
Congress leadership is - so far - not laying this defeat at the feet of Mr Gandhi, but in the organisation and campaign strategy of the party as a whole.
Personality contest?
Many Congress analysts might privately admit that Rahul Gandhi was on the losing side of an unwinnable personality contest. The biggest roadblock in their way, everyone agrees, was "Brand Modi".
"Even though the prime minister failed to fulfil the promises he made in the last election, he's still able to convince people about the policies of his government," Mr Singh says.
This is not the first time Mr Gandhi has received such a battering at Mr Modi's hands - he was all but written off after the party's worst ever performance in the 2014 elections when they won only 44 seats.
Subsequently, the party also lost several state elections and Mr Gandhi was criticised for being "remote and inaccessible" and was ridiculed on social media as a bumbling, clueless leader prone to gaffes.
He was also criticised by many for his dynastic links to the Nehru-Gandhi family and PM Modi, who comes from a humble background, has repeatedly said that Mr Gandhi had risen to the top not on merit, but because of family connections.
In private conversations, party workers describe Mr Gandhi as a "simple" man who lacks the "wiliness and cunning of his rival". So is this seen as a failure of the man or the Gandhi brand?
Gandhis reborn
But the party machinery does not blame him or his name for their loss. One party worker suggested what Mr Gandhi needs is an "Amit Shah" - a reference to the BJP party president who's helped craft Mr Modi's winning strategy and has been given credit for the party's victory - first in Gujarat and now in Delhi.
It's unlikely that they will blame Mr Gandhi - at least openly - for the debacle. If the past is any indication, they will rally behind him.
In the past two years Mr Gandhi's career graph had also begun to improve: he'd emerged from the shadows and was wearing his political skin with more ease. His social media campaigns became smarter and he began arguing convincingly about the government's controversial currency ban, a lack of employment opportunities, growing intolerance in the country and the slowdown in the economy.
He was increasingly seen as setting the agenda with a combative campaign and in December when he led the Congress to victory in important state elections in Rajasthan, Chhattisgarh and Madhya Pradesh, many said he had brought the party back into the reckoning.
And then in February, when his charismatic sister Priyanka Gandhi joined him in Uttar Pradesh, it seemed like the Gandhis were on to something.
There has long been a school of thought among some Congress enthusiasts that Priyanka might be the Gandhi to save the political dynasty. For whatever reason, she has been reluctant to take on that mantle. The siblings are believed to be close and it is unlikely she will be part of any plan that would see him pushed out. But she may well take on a bigger role in working with and supporting him.
In the end, this is being seen in Congress as a broad failure of party vision, an inability to connect with an India that Mr Modi has both defined and taken the pulse of.
"If you look at our manifesto, it's the best. The policies we had announced, the promises we made were top notch. But what we had hoped for from the voters, the support we had hoped for, that didn't happen," says state party official Virendra Madan.
Mr Madan says the party leadership in Delhi - as well as the state level - will hold meetings in the next few days to figure out what went wrong. "It's time for soul searching. To assess where we went wrong."
But, he says, that no matter how decisive the election result, there's no question of the party not standing by its leadership.
"It's not just Mr Gandhi who has lost. Lots of other leaders also did not win. And elections come and go, you win some and you lose some. Remember in 1984, BJP was down to just two seats? Didn't they make a comeback? We will also come back," he says.