The plague during Justinians reign greatly damaged Byzantium.
That was a big one. The plague of 541-542 killed maybe 1/4th of the empire. And back then those plagues kept recurring in waves for decades.
Justinian though still had some big military victories in Italy in the 540s and 550s, and Maurice had successful campaigns in the Balkans in the 590s.
The big military reverses didn’t occur until the early 600s, against Persia (although successfully repulsed by Heraclius), and the Muslim onslaught in the middle of the century.
A non-plague factor in the loss of Egypt and Syria was the Monophysite religious conflict, which alienated them from Constantinople.