I wouldn't feel safe unless I were a tank of gas away....and 20 miles ain't! We're 165 miles east of Los Angeles, and IMO that's not far enough.
In my opinion, 165 miles is more than far enough. I don't believe zombies would actually drive starting with a full tank until it hit empty. More likely, they would drive to the first likely location, probably within a mile of their starting point. The survivors would then go a mile or two from that point. Refugees would drive out of the city until they literally ran out of gas and in some cases blocked their lane. I think you are more than far enough for practical purposes, if for no other reason than because as you get further from LA, you get closer to Vegas or some Arizona city.
As for 20 miles not being enough, you are almost certainly right. However, I have other considerations to balance. I suspect that 20 miles is far enough to substantially reduce both the number of zombies and their health when they arrive, and I am potentially betting my life on that.