My husband's sis lives in SLO, so we visit there several times a year. It is a stunningly beautiful little town. Very charming vernacular architecture (each charming little house complete with a charming little garden), a beautiful though "boutique-y" downtown area, and beautiful hills in the distance. Wonderful climate. It is brimming with culture (if you don't mind it being at the college level) and educated people. Extremely liberal area, just like San Francisco. Well worth checking out, if you can stomach the endemic liberalism. (Of course, if you can't stomach the rampant liberalism here, why would you even consider moving to CA?) I disagree that it is cheaper, though. We are in Santa Rosa, near SF, and SLO costs seem to be higher than here. Also they have no shopping to speak of -- if you want to go to Home Depot or any other large store, you'll have to drive at least half an hour.
The scary thing after reading your post is remembering that SLO is the most conservative place that I have ever lived in.
I think it was the first place that I found people that voted Republican that were not my family.