I've been in two communities where the local businesses were pissed about Wal-Mart and K-Mart moving in.
A few years later, business was thiving in both towns. When people spent less on closthes, etc, they had more money to spend on other things.
And folks who want quality WILL pay to get it.
My experience has been nearly identical to yours, although in one community the local independent businesses were bending over backwards to get WalMart in - they hd seen how similar businesses in other communities had thrived once WM moved in.