The Constitutional right to associate, includes the right NOT to associate.
That cuts both ways - the fraternities have the freedom to choose who they will associate with, but the university (a private university) has the right to choose which organizations they will and will not recognize.
Not saying this is the right move (it’s not), but it’s the university’s choice.