Before they got Bryce Harper a few years ago, the Phillies were usually in the bottom third of the major leagues in payroll. They never seemed to be in the hunt for the top free agents at any given time -- the Gerritt Cole, the Alex Rodriguez, the Manny Ramirez, the Jason Giambi, the Max Scherzer, etc.
Sometimes size doesn't mean as much in the context of specific sports. Montreal is one of the biggest markets for hockey but failed miserably as a MLB market. St. Louis is a huge baseball market but can't keep an NFL team for its life. And for years, Los Angeles seemed to be a giant market for every sport EXCEPT the NFL.
Detroit would love to have an NFL team...