You are SO right about that! My 3rd, now 2nd (my eldest died in Dec) was a cat I took in from my backyard. He came with his pal (which my mom took). They were too clean and friendly to be strays so I knew they were abandoned in some way.
Anywho, before they were rescued I used to feed them crunchies outside. One night I went out to have a butt and there were 2 raccoons eating their crunchies and those 2 sitting right there, just looking at them!
Of course I freaked out and scared the raccoons away, but they really had NO idea what danger they were in.
Mimi (I thought he was a she when they were in the backyard, hence the name)has absolutely NO desire to go outside. She is very skittish as well. The vet estimated her to be between 9-11, poor thing has 4 teeth in her mouth, including 1 fang, which she gets me w/ all the time!