If they are deliberately skewing their models by introducing spurious factors, there will be an outcry in the scientific community sooner or later. It would be a huge scandal comparable to the N-ray scandal.
That would depend on what you mean by "spurious". The IPCC has described its modeling procedure pretty well; the upper-range estimates have been obtained by taking out some negative feedback factors in some models. They evaluate a range of models to come up with a range of climate scenarios, and then the evaluate the consequences of those possibilities.
The media alarmists tend to glom onto the worst-case scenarios and present them as if they were the likeliest scenarios -- which isn't the case. But "moderate changes" don't make stories that lots of people will read. So there really isn't a scandal here (nor is there a deliberate attempt to skew the science) -- the name of the IPCC game is to evaluate the broadest possible set of circumstances, and then propose various ways to address different sets of circumstances.
Of course, members of the IPCC community have seized upon particular aspects of the reports and used them to push their own agendas. Not much can be done about that.