That's an incorrect interpretation of scripture. Colossians 2:16,17 reads:
Col 2:16,17 Then do not let anyone judge you in eating, or in drinking, or in part of a feast, or of a new moon, or of sabbaths which are a shadow of coming things, but the body of Christ.
The first thing to notice is that Paul, in inspired scripture, says that holy days and sabbaths ARE shadows of things to come. ARE, not WERE.
The second thing to notice about the verse is that the "judging" wasn't about WHETHER those in the Colossians church were celebrating holy days, but HOW. The Colossians WERE celebrating God's holy days. And many of the Colossians were gentiles, not jews.
In Col 2:14, Paul talks about the fact that Christ saved us with His sacrifice, "Blotting out the handwriting of ordinances that was against us". In continuing that thought, Paul is telling the Christians there that they should let no one judge them for keeping or not keeping the old ways (Col 2:16-17). Logically, if no one is going to judge us on keeping the old holy days, then there can be no requirement to keep them.