The announcement, made by his spokesman at a press conference in Moscow where Duterte was on an official visit, fulfils an often-repeated warning by the president that he would enforce military rule to quell security threats.
Duterte placed all of the southern region of Mindanao, which makes up roughly one third of the country and is home to 20 million people, under martial law, spokesman Ernesto Abella said in the nationally televised briefing.
He said martial law would be in place for 60 days, in line with constitutional limits on the use of military rule.
Martial law is particularly sensitive in the Philippines because it was used by dictator Ferdinand Marcos to remain in power during his two-decade reign, which ended in 1986 with a "People Power" revolution.
Duterte met with Russian President Vladimir Putin late Tuesday at the Kremlin, prior to his return to the Philippines, and explained his abrupt departure.
"Units of the Islamic State Group are occupying a province and there are clashes, a military operation is still ongoing. Unfortunately, I must go there now," he said at the start of the meeting.
The announcement of martial law came after security forces battled dozens of IS-linked gunmen in a Marawi, a city of about 200,000 people in Mindanao, on Tuesday.
Marawi is about 800 kilometres (500 miles) south of Manila, the nation's capital.
"Our country needs modern weapons... to fight against the Islamic State group we need weapons," Duterte said in Moscow.
Defence Secretary Delfin Lorenzana said one policeman and two soldiers were killed in the clashes, which began when security forces raided a house where they believed Isnilon Hapilon, a leader of the infamous Abu Sayyaf kidnap gang and Philippine head of IS, was hiding.
Photos posted on social media by residents showed the gunmen walking through the streets of Marawi and placing black flags that looked similar to those used by IS.
Lorenzana said the gunmen, who were believed to number more than 100, had occupied a hospital and a jail, and burnt down a buildings including a Catholic Church.
He said many were hiding in buildings as snipers, making it difficult for security forces to combat them.
The Abu Sayyaf, based on the most southern islands of Mindanao, has kidnapped hundreds of Filipinos and foreigners since the early 1990s to extract ransoms. The United States lists it as a terrorist organisation.
Abu Sayyaf militants beheaded an elderly German man early this year and two Canadians last year after ransom demands for many millions of dollars were not met.
It has also been blamed for the country's worst terrorist attacks, including the 2004 bombing of a ferry in Manila Bay that claimed more than 100 lives.
The US State Department offers a $5-million bounty for Hapilon's capture.
Security analysts say Hapilon has been trying to unite Filipino militant groups that have professed allegiance to IS.
The Maute group has engaged in repeated deadly battles with the military over the past year in rural areas around Marawi.
Muslim rebels have been waging a rebellion since the 1970s for an independent or autonomous homeland in Mindanao, with the conflict claiming more than 130,000 lives.
The main Muslim rebel groups are involved in peace talks with the government.
But the Abu Sayyaf, Maute and other hardline groups are claiming they want to set up an Islamic caliphate in the south for IS, according to security analysts.
The fighting on Tuesday came six weeks after the military foiled a mass kidnapping attempt by the Abu Sayyaf on the central island of Bohol that is popular with foreign tourists.
The US and other Western governments warned this month that terrorists were planning to kidnap foreigners in other tourist hotspots in the western and central Philippines, adding to longstanding advisories of abduction threats in Mindanao.
Duterte, 72, known internationally for waging a war on drugs that has claimed thousands of lives, had said on many occasions since becoming president last year that he was prepared to declare martial law nationally as well as in Mindanao.
However Abella and Lorenzana emphasised on Tuesday that martial law on this occasion remained restricted to Mindanao.