Greek was spoken among the Roman elite the way French was the language of the European aristocrats from England to Moscow a few centuries ago.
In the Western part of the Roman Empire at this time Latin was spoken more because the local populations were never part of the Hellenistic world and being poorer than the Greek east-the least educated lower Roman classes were sent who were not as well educated in Greek (their famalies could not afford to own or hire Greek educators).