I guess one of the reasons I found his use of the word offensive is because I've heard the mobsters on The Sopranos sometimes call someone a mutt. In that context, it seems to be one of the worst insults you can hurl.
Of course that's TV. I don't really know any mobsters.