David Crockett died on March 6, 1836. Over 60 years later would place the Wig Party writing at around 1896. The Wig Party gasped its last breath in 1856, along with the advent of the Republican Party.
So, your premise is wrong. Want to back up your second premise of their not being no congressional record of said comments?
It appears I am wrong to attribute the story to the Whig party, since their last national candidate was in 1956.
However, If he had actually made the speech to Congress as claimed, it would be in the congressional record of that time, and it is not.
In congress, Crockett was best known for his support for giving away public lands to squatters and homesteaders.