You can do it earlier, but in addition to the regular taxes owed, you'll get hit with an additional 10% penalty for, uh, premature withdrawal.
Seems nobody likes premature withdrawal