How is using armed Federal soldiers to end Freedom of Association protecting the rights of its citizens?
If this were a private pool, or a private softball league on a private field, or a private country club, then I'm prepared to bring Freedom of Association into the argument.
If it's a public pool supported by public funds, I don't believe Freedom of Association permits denying use by Jewish citizens; I don't believe Freedom of Association permits a municipal softball league to deny participation by a Buddhist team; and I don't believe Freedom of Association permits a public school to deny access to black students.