The scattering is the issue for me. If I have a child in the house I want to make darn sure where my shot is going. In a confined space the .357 or .45 is just as loud, and with the safety slugs you don't have to worry about over penetration.
No question about the scattering. I would be leery of having a shotgun in the house.
My mother and uncle grew up in Phoenix when a loaded shotgun was ALWAYS sitting in its wall cradle...ready for use.