If the law were written according to libertarian theory, you would be correct. But the law isn't.
The law distinguishes between burglary and robbery by using force or threat of force as the distinguisher. If I were to sneak into your house and steal, that would be burglary. If I were to hold you at gunpoint and steal, that would be robbery.
In many places, the law does not allow you employ deadly force in defense of your property, but only in defense of a life.
Then the law should be changed. Or do you think the distinction makes for less crime? Which view would be the greater deterrent to crime?
Gee, that's funny, I got the definition of "burglary" from the Georgia Code, Annotated.