If your argument is that landlords have the right to reject tenants for moral reasons, then that argument is lost already. You can't discriminate against someone based on their religion, marital status and the like. It's true that discrimination against blacks was a unique and pervasive wrong that need to be corrected by unprecedented means. But that doesn't mean it's okay to discriminate against a Buddhist.
The very idea that a private business is a public accommodation is problematic, in my opinion. It certainly isn’t libertarian to support such a concept.
Unless that freedom to associate involves religion or the lack thereof, sex, marital status, race, or sexual orientation, in which case you don't. Can you name any person that does not have a religion or lack of one, a sex, a marital status, a race, or a sexual orientation? With whom do we have the right NOT to associate with then, if all are protected?