Been there, done that. Never had a second thought.
Granted, my solution is simple but unworkable for the insecure, for those with a cell phone constantly in their ear, or who rush to answer the phone on the first ring... as if a telephone is meant for the convenience of all others, rather than oneself.
I simply have my answering machine on all the time. I can hear the caller. If I know them, I answer. If I don't, the system just dutifully tolerates another non-message.
We have caller ID -- don't recognize the number -- don't answer. Can't believe the amount of "800" calls we get, and we are on the "Do Not Call" list.