I think they are supposed to be Jews.
Interesting. I was thinking they were the upperclass (many of them are reading books (Bible? - or maybe the Torah), and looking at the more underclass to the right in their simple robes. Or monks in robes? I suppose folks have done their thesis papers on this painting!