The destroyer Takanami with some 200 sailors left Japan's main naval base in Yokosuka, near Tokyo. Its main task is primarily to gather intelligence in the Gulf of Oman and nearby waters.
Prime Minister Shinzo Abe called the waters "a lifeline that supports the lives of the Japanese people." Thousands of Japanese ships ply the route every year, transporting 90% of the country's oil supply, he said.
"Your mission of information gathering is an extremely significant one that directly affects people's lives," Abe told the troops in a televised sendoff ceremony at Yokosuka.
Sending warships to areas of military tension is a highly sensitive issue in Japan because its pacifist post-World War II constitution limits the use of force by the military strictly to self-defense. Abe has gradually expanded Japan's military role since he took office in late 2012.
Under the plan, the destroyer and some 200 Maritime Self-Defense Force troops will join a pair of P-3C reconnaissance aircraft that were operating in the same area from a base in Djibouti in the Horn of Africa.
The Abe government is expected to authorize troops to protect Japanese commercial ships in case of danger, a plan that has faced opposition from pacifist groups.
Despite being a U.S. ally, Japanese troops are not part of a U.S.-led coalition protecting Middle East waterways, because Tokyo wants to maintain neutrality in a show of consideration to Iran.
Japan, which has friendly ties with Iran, seeks to serve as a mediator between the Washington and Tehran and play a greater role in restoring regional stability.
In June, a Japanese-operated tanker was attacked in the Gulf of Oman. Washington said Iran was responsible and urged Japan to join the U.S.-led military initiative.