According to your theory we should have the high number of deaths each year for the flu, but we do.
I'm assuming you meant to say that ‘we should not have the high number of deaths each year from the flu, but we do’.. It's a very good point, but we have to consider that ‘less virulent’ doesn't mean non-lethal. It is likely that we would have a much higher number of influenza-associated deaths if a more lethal strain were to emerge.