Your sense of smell is tied the only part of your brain exposed, I have no problem believing that in the general case if you lose your sense of smell, you likely will die earlier... especially if no specific onset even can be tied to the smell lose.
Like all things in statistics, there are always cases outside the average, but given losing your sense of smell is the result of damage to the brain, even if it is just 1 part of it, a generally higher likelihood of death is not difficult to believe.
If you were a smoker it could be from damage to the nasal passages and have nothing to do with the brain. Of course, smoking is also a cause of early death.