It doesn’t have to be inside your house, your property is enough. If the guy was shooting at you from the neighbors back yard, I guess he’s in the clear?
If a guy was shooting at me from a neighbor’s back yard, he is putting my life in immediate jeopardy and I would be justified in shooting him.
If a guy is in a neighbor’s back yard and I shoot him while he is unarmed, regardless of what he did in my house - I go to jail for murder or manslaughter.