"Many of them buy houses a few miles away from the border, beside German unemployment rate is much lower than Polish."
The overall unemployment rate might be lower, but it isn't lower in this area, which is the former East Germany. The rush of investment and new jobs just hasn't happened, at least not nearly to the degree predicted. As a result, towns are emptying out as people relocate to other areas to find work. Or, at least that is my understanding.