Diocletian effectively destroyed the last remnants of the local self-government characteristic of ancient Mediterranean city-states. Any local monumental building stopped and even maintenance as tax revenues were siphoned off to pay the Roman Army.
And the army itself was now divided into two different components: the limitanei who were given land in exchange for military service in frontier garrisons and the "presential" army (the "Army in the Emperor's Presence") who had more cavalry and were kept under the watchful eye of the emperor lest they revolt. This change was mainly driven by the political threat caused by constant army revolts and forced the Roman Empire into a "defense in depth" strategy where any large outside incursion that got past the frontiers would be met by the presential army. If that army were destroyed, however, the Empire was screwed by not having enough mobile forces. This is essentially what happened at Adrianople in 378.
During its run, the Empire generally relied on hired guns to collect taxes, who were, uh, very thorough, because they got a cut, rather than have a bunch of undermotivated lifetime corrupt bureaucrats.
Diocletian ended the empire's Crisis of the 3rd Century, required supplicants to conduct their audience face down with their arms straight over the direction of their heads (this is where the Popes picked up that practice afaik), divided the empire into four parts, set up the first system of succession, then put it to the test by retiring and compelling his eastern colleague into stepping down at the same time.
Lapses into seeing internal rivalries as the more important threat, at bad moments when the external threats were much more serious, turned out to be the biggest problem faced during all of Roman history.