There wasn’t a lot of money in the area just outside where I grew up. What was just recently the poorest place in Michigan isn’t Detroit, Flint, or Benton Harbor, but my own Roscommon. Roscommon has its issues, especially with the housing crash and tourism hits, but is also a lot safer than these cities. An average person isn’t going to get jumped, robbed, or attacked there for being in “the wrong area”. You have to look for trouble to find it.
Not too far behind is Lake County. The only difference is you will get jumped if you dare go to parts of Baldwin, the County Seat. For some reason minorities congregated there even though there wasn't much to steal and destroy to begin with.