Lots of people would have disagreed with you. In the Middle Ages, an opponent of the local ruler, or even sometimes a person who had fallen into his disfavor, had only to flee across the nearest border and he could gain freedom and even allies to revenge himself.
Under the Empire, there was nowhere to flee for those who were out of favor of the Emperor. In essence, the Empire was a political monopoly, feudalism had some features in common with free-market economies.
Feudal societies are not usually very progressive, but they are also inherently sloppy in their opporession. This sloppiness allowed for many people to lead a relatively free existence, as compared to a modern totalitarian society.