The militants launched a targeted bomb and gun attack on the al-Rowda mosque in Al-Arish city during the Friday prayers, the state-run MENA news agency reported.
After the bomb ripped through the mosque, the gunmen on four off-road vehicles opened fire on the worshippers who tried to escape from the site after the explosion, it said.
Pictures from the scene show rows of bloodied victims inside the mosque.
The blasts from improvised explosive devices caused considerable damage to the mosque, Ahram Online said. Speaking to state-run Masriya TV station, Egyptian health ministry spokesman Khalid Mujahid described the incident as a "terrorist attack."
One report said the target appeared to be supporters of the security forces who were praying at the mosque.
Local people are also quoted as saying that followers of Sufism, or Islamic mysticism, regularly gathered at the mosque.
Islamist jihadist groups, including so-called Islamic State (IS), see Sufis as heretics.
About 50 ambulances were rushed to the attack site to shift the injured to hospitals.
No one has yet claimed responsibility for the attack and there is no word yet on what happened to the militants involved. However, it bears the hallmarks of an attack by ISIS, reports said.
There have been regular attacks blamed on militants on the Sinai peninsula since the January 2011 revolution that toppled former president Hosni Mubarak, but this is the deadliest assault of its kind.
The attacks targeting police and military increased after the ouster of Islamist ex-president Mohamed Morsi in 2013 by military following massive protests against his rule.
Over 700 security personnel have been reported killed since then.
The military has launched security campaigns in the area, arrested suspects and demolished houses that belonged to terrorists, including those facilitating tunnels leading to the Gaza Strip.
The Egypt government has announced three days of mourning, even as President Abdel Fattah el-Sisi chaired an emergency meeting with officials to review security situation after the deadly attack.
Egypt has witnessed a series of terror attacks this year claiming scores of lives.
On May 26, gunmen attacked a bus carrying Coptic Christians in central Egypt, killing at least 28 people and wounding 25 others.
On April 9, two suicide bombings at Palm Sunday services at churches in the northern cities of Alexandria and Tanta left 46 people dead.
- - - - - - - - - Advertisement- - - - - - - - -