The abrupt decision to shift the marathons and race walks was announced almost two weeks ago by the IOC.
Tokyo Governor Yuriko Koike is angry about it. Her allies say no change is needed and have raised questions about who will pay if the move goes through, and have not ruled out a lawsuit to recover damages.
Taro Shirato and Hiroshi Yamada, members of Koike's political party in the metropolitan legislature, told a news conference Tuesday that moving the marathon would cost at least $34 billion yen, about $310 million.
The IOC said it's making the change, thinking first of athletes' safety from Tokyo's blistering summer heat.
Koike's allies offered a different take. Koike is one of Japan's most influential politicians and just a few years ago was viewed as a potential candidate for prime minister. And she's miffed about not being consulted.
"Although they (IOC) talk about so-called athletes first, this can only be perceived as IOC first," Shirato said through an interpreter.
"You get the sense that no considerations have been made for the athletes," Shirato added, "or the spectators who had already bought their tickets and who were looking forward to these events, or the potential spectators who will be cheering on the streets, and also to the people involved in the operation."
Don't expect the IOC to budge. It has inspectors in Tokyo this week looking at preparations with the Olympics opening in just under nine months on July 24.
IOC member John Coates heads the team and is an ally to President Thomas Bach. He has said repeatedly the IOC does not intend to change its plans, and has told that to Koike.
The IOC fears worldwide television audiences might see a repeat of the recent world track and field championships in Doha, Qatar, where 28 of 68 starters failed to finish the women's marathon and 18 of 73 men failed to complete the course.
The races started at midnight in Doha with TV showing runners collapsing on the course. The scenes apparently shocked IOC President Thomas Bach.
Yamada acknowledged the heat posed a risk. He said Tokyo has proposed moving the start to 5 a.m., which is mid-summer sunrise in Tokyo. Last week city officials also floated the idea of a 3 a.m. start.
Estimates suggest the temperature would be 27C (81F) at 5 a.m., and would be 25.4C (78F) in Sapporo for a 7 a.m. start. The starting temperature in Doha for the women's marathon was 32.7C (91F).
Yamada described the starting temperatures in Tokyo and Sapporo "on a par."
"We do recognize and understand that the heat is a very important factor, but we do not believe that at this moment it represents an overly excessive risk."
Tokyo's soaring costs are also a major issue.
A government audit report last year said Tokyo was spending about $25 billion to organize the Olympics, all of which is public money except for $5.6 billion from a privately financed operating budget.
Tokyo said in its bid in 2013 that the Olympic would cost $7.3 billion.
Yamada was asked who would pay for the increased costs.
"In the event this is changed to Sapporo, then I believe the citizens of Tokyo will not be convinced they need to pay. What I can say is that the Tokyo Metropolitan Government should not be the one to pay."
Asked if the Tokyo government might sue for damages, Yamada hedged.
"That's very difficult to respond to, but I believe in terms of discussions we need to clarify the legal context."
Tokyo's organizing committee president Toshiro Mori, a former Japanese prime minister, seems to have sided with the IOC and not with Koike. He suggested a few days ago that it was a done deal.
"Can we say no to the plan that the IOC and International Association of Athletics Federations already supported?" Mori said. "It's not a question of good or bad, but we just have to accept it."
He also said cost was a major issue in moving the marathon.
"Our overall cost has become a humongous amount, so it would cause us pain if the cost is added to our bill," Mori said. "So I mentioned that to Mr. Coates, and he said he will look into it. We won't be able to pay if it's a significant damage to our finances. I have reminded him of that."