The grenade was found during the night near buildings housing 170 people in the town of Villingen-Schwenningen. Its pin had been pulled out but the explosives failed to detonate.
Justice Minister Heiko Maas said it was a new level of “hate and violence”.
There were 1,005 attacks on refugee homes in Germany last year – five times more than in 2014.
Some of the migrants at the Villingen-Schwenningen hostel were evacuated while bomb disposal experts carried out a controlled explosion.
Police spokesman Thomas Kalmbach said it was “just luck” no-one was hurt.
Germany’s justice minister said the incident marked a “new level of hate and violence”
Map of Villingen-Schwenningen
Officers said the grenade still contained its explosives but it was not clear whether it still had a detonator.
Mr Maas said he was summoning his regional state counterparts to crack down on hate crime.
“Grenades are already flying towards refugee homes. We can’t wait until there is someone dead,” he said.
Spike in attacks
News of the grenade incident came as new figures showed five times more attacks were carried out on migrant hostels in Germany last year than in 2014.
The total for 2015 was 1,005, compared with 199 in 2014, the police report said. Far-right activists are suspected in 90% of the cases.
Last year a record 1.1 million people sought asylum in Germany – many from war-torn Syria.
Many local authorities have struggled to house them.
Germany is expanding its list of safe countries of return, hoping to curb the influx.
The governing coalition plans to declare Morocco, Algeria and Tunisia safe countries of origin, making it easier to send migrants back, said Vice-Chancellor Sigmar Gabriel.
Migrant arrivals in 2015 – map
Last year Germany did the same for several Balkan nations – including Albania, Bosnia-Herzegovina and Kosovo – to cut the large numbers of migrants claiming asylum. Very few of their applications are granted.
The police say the biggest rise in attacks on migrant hostels last year was registered in the mainly industrial state of North Rhine-Westphalia.
State Interior Minister Ralf Jaeger said “investigators have noticed a marked increase in aggressive language” towards migrants on the internet.
Most of the thousands of migrants arriving daily on Greek islands hope to get asylum in Germany.
Where Europe is failing on migrants
- The 28 member states have not agreed on an EU-wide mechanism for relocating migrants, to ease the burden on Greece and Italy; only small groups have been relocated so far – and several states in Central and Eastern Europe refuse to accept migrants
- The Schengen agreement on freedom of movement is in jeopardy – Hungary fenced off its borders with Serbia, Croatia and Slovenia; meanwhile Germany, Austria, Denmark, Sweden, Norway and France have also reimposed border controls
- The Dublin regulation, under which refugees are required to claim asylum in the member state in which they first arrive, is not working effectively; countries are no longer sending back migrants to their first point of entry to the EU
- Thousands of migrants – many of them Syrian war refugees – still arrive daily from Turkey
Processing of asylum applications is slow and there is a big backlog – so reception centres are overcrowded
- Germany – the main destination for migrants – is rethinking its open-door policy, partly because of outrage over assaults on women in Cologne at New Year