According to results announced by the Election Commission on Wednesday, the opposition Congress party was a clear winner in Chhattisgarh state, and fell one seat short of a majority in both Rajasthan and Madhya Pradesh. With a regional party offering support, the Congress party is set to rule all three states, replacing Modi's Bharatiya Janata Party.
Voting in the three states was spread over the past two weeks.
The setback to the BJP is expected to revive the political fortunes of the Congress party under Rahul Gandhi, the 48-year-old scion of the Nehru-Gandhi family, who took over as party president from his mother, Sonia Gandhi, a year ago. He had campaigned extensively for his party in the three states.
India's national elections are due by April. Modi came to power in 2014 after his BJP wrested power from the Congress party.
Modi is still the front-runner for the next elections. But his party's defeat in the three state polls is expected to bring disparate opposition parties together with the Congress party as the main threat to Modi.
The BJP attributed the defeat in two of the three states to an anti-incumbency factor, as it had ruled Madhya Pradesh and Chhattisgarh for the past 15 years without a break.
Rahul Gandhi told reporters that people voted against the BJP because it failed to create millions of jobs for young people as promised by Modi and deal with unrest by poor farmers across the country over the government's failure to provide them higher prices for their produce and a waiver on their farm loans to alleviate hardships.
In a tweet, Modi congratulated the Congress party for its victory and said he accepted the outcome with humility.