The rocket attack threatens to undermine Egyptian-mediated efforts to cement a deal that the Gaza Strip's Hamas rulers hope will ease a crippling Israeli-Egyptian blockage of the crowded territory.
No casualties were reported and no Palestinian group claimed responsibility for the rockets, though they appeared to be in retaliation for the deaths of the protesters.
Tens of thousands of Palestinians rallied in the Gaza Strip on Saturday to mark the anniversary of their mass protests along the Israeli border.
Most demonstrators kept their distance from the border, though small crowds of activists approached the perimeter fence and threw stones and explosives toward Israeli troops on the other side. The forces responded with tear gas and opened fire, killing four Palestinians and wounding 64.
Hamas had pledged to keep the crowds a safe distance from the fence to avoid inflaming the political atmosphere during negotiations of a possible easing of the blockade.
Hamas officials say that Israel is offering a package of economic incentives in exchange for calm along the volatile border.
Khalil al-Hayya, a senior Hamas official, said the group had received "positive signs" from the Egyptians. He added that the Egyptian team was to return to Israel on Sunday to continue the talks. "We will continue our marches until all our goals are achieved," he said.
Saturday's protest came at a sensitive time, with Israel and Hamas, bitter enemies that have fought three wars and dozens of smaller skirmishes, both having a strong interest in keeping things quiet.
Israeli Prime Minister Benjamin Netanyahu is seeking his fourth consecutive term in April 9 elections, but is facing a serious challenge from a group of ex-army chiefs who have criticized what they say is his failed Gaza policy. With a lack of alternatives, Netanyahu has been forced at times to rely on Hamas to maintain stability along Israel's volatile southern front.
In the final stretch of the campaign, Netanyahu needs to keep the Israel-Gaza frontier quiet, without seeming to make concessions to Hamas. Netanyahu took heavy criticism this week for what was seen as a soft response to renewed rocket fire out of Gaza.
Hamas, meanwhile, faces growing unrest in Gaza as a result of worsening conditions after more than a decade of an Israeli-Egyptian blockade. The two countries imposed the blockade in 2007 after Hamas, an Islamic militant group that seeks Israel's destruction, seized control of Gaza from the internationally recognized Palestinian Authority.
The blockade has helped drive unemployment over 50 percent, led to chronic power outages and made it extremely difficult for Gazans to travel out of the territory.
Earlier this month, Hamas violently suppressed several days of public protests, staged under the slogan "We want to live," over the dire conditions.
Speaking on the group's Al-Aqsa TV station, Hamas' top leader in Gaza, Yehya Sinwar, praised the protesters. "With this big turnout, our people say, 'We want to live!"
His use of the protesters' slogan appeared to be aimed at diverting the recent criticism of his group. Hamas blames the blockade and punitive measures by its West Bank-based Palestinian Authority for worsening the living conditions.
The fence protests, which began exactly a year ago, have been aimed in large part at breaking the Israeli-Egyptian blockade on Gaza, but haven't delivered major improvements.
Saturday's demonstrations were held at five rallying points along the border with Israel. Dozens of volunteers in fluorescent vests were deployed to restrain demonstrators, and cool rainy weather also appeared to affect enthusiasm.
But as the crowds swelled throughout the afternoon in response to Hamas' calls for a large turnout, dozens of protesters approached the fence, unfurling Palestinian flags and throwing rocks and explosives toward Israeli soldiers. The Israeli forces responded with tear gas and live fire.
The Israeli military estimated 40,000 Palestinians were gathered at the marches.
"The rioters are hurling rocks and setting tires on fire. In addition, a number of grenades and explosive devices have been hurled at the Gaza Strip security fence," it said in a statement.
In a statement, Prime Minister Netanyahu praised the army's preparation and performance in maintaining "calm."
Gaza's Health Ministry said that a 17-year-old protester died immediately after being shot in the face in east Gaza City. In the evening, the ministry said another 17-year-old died hours after being shot in the chest in a different protest location.
A third teenager, also aged 17, succumbed to his wounds and died in the late evening. A 21-year-old Palestinian also died around dawn after sustaining injuries in overnight protests before the main demonstration.
While bloodshed was not avoided, it was far less than previous high-profile protests. Over 60 people were killed during intense protests on May 14, the day the U.S. moved its embassy to Jerusalem.
As Saturday's protest was winding down, organizers vowed to continue the marches and said they would gather again as usual next Friday.
The military released video footage showing large crowds of protesters gathered near the fence and hurling objects.
In one scene, a group of activists went up to the fence and hurled stones at the other side. In another scene, a youth could be seen trying to pull apart barbed wire along the fence.
The army also said it caught two young Palestinian children who had tried to cross the border with a knife. The children were returned to Gaza through a border crossing.
Earlier on Saturday, Gaza health officials said Israeli troops shot and killed a 21-year-old Palestinian man near the perimeter fence, hours before the mass rally.
The army said about 200 Palestinians "rioted during the night along the fence" and that the army used riot dispersal means against them.
The marches near were initially organized by grassroots activists who were calling for a mass return to ancestral homes in what is now Israel.
Hamas quickly took the lead in the protests, using the gatherings to call for an easing of the blockade.
The border marches routinely ended in confrontations, with some of the Palestinian demonstrators burning tires, hurling fire bombs or setting off explosives and Israeli troops firing live rounds and tear gas.
According to a Gaza rights group and a count by The Associated Press, 196 Palestinians were killed in the demonstrations over the past year, including 41 minors, and thousands were wounded by live fire. An Israeli soldier was also killed in the context of the marches.
Israel says the army has been defending the border. The army accuses Hamas of using the large crowds as cover and encouraging demonstrators to hurl explosives, incendiary balloons and grenades across the border. But Israel has come under heavy international criticism for the large number of unarmed people who have been harmed.
Egypt has repeatedly tried to broker a cease-fire between Israel and Hamas, stepping up efforts in recent days after a Gaza rocket struck a house in central Israel earlier this week, injuring seven Israelis and threatening renewed escalation.