I wouldn't recommend it either.
You really have to watch Season 1 first to understand the relationships between Jack and Kim, Jack and Nina, Jack and David Palmer, etc.
IMO, Season 2 is the best, followed by Season 4. But if you skip Season 3, it'll leave gaps in your understanding of Season 4.
In short, you could skip around if you want, but you'll miss some background on the relationships between characters.
Last year I started out by watching Season 5. Then I bought Season 1 and watched the whole thing. Throughout the year, I'd watch other seasons on A&E. Not knowing the whole background didn't interfere with my enjoyment of Season 5. It was still exciting.