George W. Bush retreated to his father's waterfront home in Maine (which was actually from Barbara's family I think) and caught grief for it or he took off for a really low-key ranch of his own in the prairie/scrubland of Texas . . . and caught grief for it.
Obama spends millions of taxpayer dollars for prolonged visits to the most elite resorts and the media has no comment on it. None.