The nation's Meteorological Agency issued a special weather warning for what it called “historic” rains.
Landslide warnings have been issued in over a quarter of the nation's prefectures.
The agency also warned of rising rivers and strong winds.
National broadcaster NHK said at least 62 people were dead and 44 were still missing.
The Fire and Disaster Management Agency said by Saturday morning, more than 1.6million people had been ordered to evacuate their homes.
A further 3.1million more were advised to leave.
Helicopter footage showed people on their roofs waving for help in Kurashiki.
The footage also showed Japanese soldiers rescuing children by boat from a flooded river in Hiroshima.
Chief Cabinet Secretary Yoshihide Suga said around 48,000 police, firefighters and members of Japan’s Self-Defence Forces were responding to appeals for help.
The agency said in Motoyama, a town on Shikoku island, around 373 miles from the capital Tokyo, a resounding 23 inches of rain fell between Friday and Saturday morning.
An official at the Japanese Meteorological Agency said: "We've never experienced this kind of rain before.
"This is a situation of extreme danger."