This is their own fault.
Even when I buy a rental, much less my own home, I visit several times during the day, cruising the neighborhood, looking for potential drawbacks. I also visit at least twice at night, specifically looking for nearby parks and corners where "young entrepreneurs" set up shop, etc. Look for graffiti, etc. And, if the price is right, look over that gift horse with extra care.
Of course, in CA they may have a cause of action against a real estate agent, or the previous owner who should have disclosed this.
You mentioned some reasoned dynamics there. I wouldn’t disagree with you. Other than the things I mentioned, I know very little about the transaction’s dynamics. Couldn’t tell you why this didn’t show up before the purchase.
It may be that the half-way house came into being after the purchase. I’m not privy to all the details.