If you're not concerned with the other person's welfare, then the only thing preventing you from robbing them is the fear of getting caught.
What are you, like 8 years old?
One can refrain from robbery without being particularly concerned about the welfare of another. Take you for instance. I don't give a crap about your welfare. Honestly if you were on fire it's only 50/50 whether I'd p****** on you to put it out.
But that doesn't mean I reserve to myself the right to rob you.
If you believe this to be true you truly have a dim view of your fellow man.
I'm not concerned for the drug addict's welfare to the point that I want my government to take my tax dollars to help feed his habit that he created via an individual choice.
Yet I have a highly developed sense of right and wrong and wouldn't even consider committing any crime against that same person.
You are attempting to defend imposing your morality on others and that position is not defensible in a free society.