Actually it is. But I doubt they stayed home. They voted for the other guy.
Apparently, looking at the vote totals, they did stay home. Obama is down something like 10M votes from 2008, but Romney underperformed McCain's 2008 numbers by 3-4M. If Romney had gotten all of the McCain votes, he would have won.