The Building and Wood Workers' International has been critical for several years of workers' rights at Olympic venues. In face-to-face meetings and in writing it has asked local organizers, Tokyo's municipal government, and the Japan Sport Council, for outside inspections of construction sites and the right to interview workers.
"However, our attempts to achieve justice for complainants have been consistently rebuffed by the three implementing agencies," the international union said in an open letter this week to Bach.
The union asks the International Olympic Committee to "intervene and directly address human rights abuses when local organizing committee fail to do so."
The BWI, headquartered in Geneva, Switzerland, only an hour's drive from the IOC's main base at Lausanne, was formed in 2005. Its website says the BWI represents about 350 trade unions and around 12 million members in 127 countries.
The BWI alleges health and safety violations and has been critical of the treatment of migrant workers in Tokyo, the use of subcontractors, and wages. It has also complained about grievance procedures.
"We are writing this 'Open Letter' to ask for your immediate intervention in the preparations for the Tokyo 2020 Summer Olympics to prevent future worker rights violations," the letter said.
The letter says three workers have died during the construction process, and it also alleges that "venues have been built using tropical rainforest timber from companies with a documented history of indigenous and worker rights violations."
"Given all our efforts to cooperate with local organizers, we must conclude that they have no intention of respecting international labor standards. What does that mean for the IOC?"
The letter references the UN Guiding Principles on Business and Human Rights, which spell out practices for businesses and government. It also details human rights standards for sporting bodies like the IOC and soccer's world governing body FIFA.
Many sports bodies have signed up to the Guiding Principles. They include FIFA and UEFA, the Commonwealth Games Federation, the International Paralympic Committee, and Qatar's 2022 World Cup.
The IOC has been slower to move.
The IOC has added the UN guidelines to host city contracts that come into effect with the 2024 Paris Olympics.
This excludes Tokyo, and the 2022 Winter Olympics in Beijing, where independent labor unions are barred by China's authoritarian government.
China is at the center of human-rights issues. Intense protests in Hong Kong have centered on demands for full, democratic elections. In western China, more than a million Muslims have been detained in reeducation camps. Those released have described them as virtual prisons.
In an email to AP, the IOC said it had asked the International Labor Organization, a United Nations body, to help negotiate a settlement.
"The IOC also engaged in a number of discussions and we still hope a solution can be found," the IOC said in a statement to AP.
Both the BWI union and the ILO attended a forum given this week by the Centre for Sport and Human Rights. A BWI spokesman promised to raise the Olympics and Japan labor issues, and to also ask questions about "remedies" if solutions are not found.
Mary Harvey, the chief executive of the Centre for Sport and Human Rights, told AP several months ago that "everyone should be taking a serious look at the risks identified by BWI's report, and by everyone, I mean everyone who is a stakeholder, including the IOC, the Japanese government and construction companies."
Michelle Bachelet, the UN High Commissioner for Human Rights, in her keynote address this week at the forum noted "sporting events and their sponsors have a responsibility to uphold human rights, including with respect to workers, athletes, fans, and the local communities in areas where sport facilities are located."
Many of BWI's complaints are outlined in a report issued earlier this year titled: "The Dark Side of the Tokyo 2020 Summer Olympics."
Most of the Olympic venues are completed, or nearly complete. But the BWI has noted that workers are also in jeopardy when the games end and venues are dismantled. I
It also said there will be "growing pressure" to complete any behind-schedule venues.
A report released last year by the national government's Board of Audit said Japan is likely to spend $25 billion overall to prepare for the games. Tokyo projected total costs of about $7.5 billion in its winning bid for the games in 2013.
Organizers dispute the $25 billion and say it's about half that much. It is all taxpayer money except for $5.6 billion in a privately funded operating budget.