The Giants play in downtown SF, which is now shithole. That said, Yankee Stadium is not exactly located in an upscale neighborhood (/src), and NYC is as woke, expensive and lawless as any other city, so there has to be something else at work.
Could it be all the pot in the stands?
Taking the subway up to The Bronx and Yankee Stadium is an interesting ride. As is the 7 Train to Citi Field.