When rioters stormed Brazil’s top government buildings in January to dispute the outcome of the presidential election, many soldiers stood by as far-right protesters broke windows, defecated in offices and destroyed valuable art.

The images from Brasilia that day still haunt the left-leaning government of President Luiz Inácio Lula da Silva. He has strived ever since to ensure that military leaders defend South America’s largest democracy and stay out of politics.

The threat isn't just hypothetical. Brazil has lived through four military coups – the most recent one in 1964, followed by two decades of brutal dictatorship.

Also Read: Bolsonaro says he may return to Brazil in the coming weeks

Lula’s task is fraught. The military is filled with supporters of ex-president Jair Bolsonaro, and its role in the new government is being diminished by the day.

Lula has already tapped more than 100 civilians to replace military officers Bolsonaro appointed to key positions, and he has moved oversight of the country's intelligence agency to his chief of staff's office, among other changes.

“Lula needed to manage his relationship with the military to be able to govern, and will continue to do so,” said Carlos Melo, a political science professor at Insper University in Sao Paulo.

Melo said Brazil's military has long believed that it has "some kind of guardianship of the country’s political process," and Bolsonaro only fueled that belief.

Also Read: Brazil deluge toll hits 44 as search continues for missing

Bolsonaro, a former army captain, appointed more than 6,000 military officers to jobs across his government and revived an annual commemoration of the 1964 coup to stoke nostalgia for the days of military rule.

Although that era was marked by human rights abuses and the loss of civil liberties, Bolsonaro and many of his supporters remember it fondly as a time of strong nationalism, economic growth and conservative values. They view Lula's efforts to tame the military as heavy-handed and misguided.

“Stop looking through the rearview mirror and govern for all Brazilians,” Bolsonaro's former vice president, Gen. Hamilton Mourão, who is now a senator, said in an interview.

The most significant move Lula has made so far has been to elevate Gen. Tomás Paiva to be the army’s top commander.

Paiva, 62, has pledged to keep soldiers out of politics and to respect the results of October’s election, in which Lula beat Bolsonaro by a razor-thin margin.

Yet Paiva has also acknowledged that most the military’s leaders voted for Bolsonaro, and he lamented Lula’s victory to subordinates just three days before the new president called to offer him the promotion — comments he later said were misinterpreted.

Lula has taken various other steps aimed inoculating Brazil from the risk of another violent uprising with at least tacit support from some in the military:

— He blocked the appointment of a Bolsonaro loyalist to command the Goiania battalion, based an uncomfortably close 124 miles from the capital.

— He placed the country’s intelligence agency — formerly overseen by members of the military — under the office of his chief of staff, which is led by civilians.

— He took a symbolically important trip to the U.S., which before the election had warned Brazilian military leaders to steer clear of politics if they wanted access to arms purchases and cooperation from American armed forces.

For now, there is no evidence of another uprising being planned or of military leaders questioning Lula's orders, according to a high-ranking official in the army and a person who works closely with the defense minister, both of whom spoke on condition of anonymity because they weren't authorized to speak publicly.

Lula enlisted the military’s cooperation twice in February: as part of a massive operation to expel some 20,000 illegal miners from the Yanomami Indigenous area in Brazil’s Amazon, and to help rescue people after mudslides near Sao Paulo.

These represented early tests of the relationship between Lula and the military, and the results were very positive, said political consultant Thomas Traumann. Still, there's no guarantee of long-term stability, he said.

It remains to be seen whether military retirees and active duty service members who either took part in the Jan. 8 riots or turned a blind eye to them will receive punishment. Some analysts believe that would be important to deter future action.

One video from Jan. 8 showed policemen at the presidential palace in the rare position of barking orders at soldiers: “Lead your troops!” one officer shouted at members of the presidential guard, which is part of the army.

Another video showed dozens of rioters surrounded by police in the palace, as a general attempts to free them. “Are you nuts?” a policeman asks. “They’re in custody!”

Hundreds of civilians who participated in the riots have been jailed and dozens indicted. But service members have so far been spared. The military prosecutors’ office and the top military court have opened 17 investigations, although neither has been transparent about the process.

The incoming Chief Justice of Brazil’s Superior Military Court, Joseli Camelo, said he was encouraged recently when the army canceled a plan to commemorate the upcoming anniversary of the 1964 military coup, a dictatorship-era tradition that Bolsonaro revived.

“This is just another demonstration that the commander is aligned with all the powers towards our common challenge, which is to pacify Brazil and definitively reinforce democracy in our country,” Camelo said.

Mourão, Bolsonaro's former vice president, says the military should not spare any of its members who are proven guilty of taking part in the riots. “The armed forces are shaped to be rigorous in the investigation of disciplinary errors and military crimes,” he said.

Even before taking office in January, Lula — who served as president from 2003-2010 — knew it was essential for him to bolster ties with the country’s right-leaning military.

Some highly regarded military leaders had openly derided him before the election, and some even campaigned to reelect Bolsonaro. For months, the army permitted anti-Lula protesters who were openly supportive of a military coup against him to camp outside their barracks.

In Lula’s first two presidential terms, his relationship with the military was marked by conciliation rather than confrontation, said Fabio Victor, a journalist who just published a best-selling book on Brazil’s armed forces and politics. But Jan. 8 appears to have altered his calculus.

In contrast with Bolsonaro's administration, few members of the armed forces work at the presidential palace, Victor said. With an eye toward the future, Lula's allies in Congress are pushing for constitutional changes that would more clearly define the military's powers and limits, and his ministers are looking at overhauling military education.

“Lula today is very suspicious of the military,” Victor said.