That's right. Frank Capra intentionally set up Henry F. Potter as an evil guy yet there's a lot to like about that character in terms of thrift and self-reliance. His planned takeover of the broken-down Building and Loan seemed to be based on sound business principles, which was not the case under the slipshod management of Peter Bailey and the dim-witted Uncle Billy. George's protest about these people deserving a decent standard of living isn't a very good argument and Potter rightly rebuffs it, in my opinion.
Probably the part I dislike most about "It's a Wonderful Life" is when George Bailey (at the behest of his wife) parts with his own honeymoon money to give to those storming the BB&L. Sure, it was his own decision and not done under the strong arm of the government but he made it too easy on those people who should have abided by the rules when they deposited their money there. I think this is the kind of charity (borderline welfare?) that Ayn Rand was against.
All that said, if I just watch the film for sheer enjoyment, ignoring some of the sketchy economic lessons, it's an entertaining yarn. And George's telephone scene with Mary is a classic.
...and on a lighter note, a few years back, we had so give and take here on FR about whether Pottersville (a Las Vegas of sorts) was actually a better place than Bedford Falls. :-)