It's worth noting that according to the Saudi's complaint about this, many of these "diplomats" were in fact clerics and academics here on diplomatic passports.
They were practicing any form of diplomatic work, rather engaged in teaching Arabic language for non-Arabic speaking people as well s for preaching the Islamic religion in America.
It took the State Dept nearly 3 years post 9/11 to simply enforce our passport regulations, as it pertains to Saudis, of course.
Statement from Saudi Embassy in Washington on Expulsion of Diplomats