Read Post# 761.
Read post 760: All humans have such rights, because if you apply rights to only some people and not others, then it's no longer a principle but instead just an ad-hoc, self-serving rationalization.
It's true in one sense that my basis for morals is myself. But as I said, morality is a principle - a system or framework of judging behavior. You can't have a principle without applying it consistently in all similar contexts. This means that it must apply to everyone in a similar situation.
Is it moral to kidnap someone? Well, it's clearly wrong to kidnap me, except if I was judged guilty of a crime by a legitimate court. In that case they'd have the right to put me in jail. But to kidnap me "just because", or because I owe somebody some money, would be wrong.
You see? Your problem isn't really against a self-interested basis for morality. It's against self-serving morality. But a self-serving morality (where you don't apply the moral principles consistently) is a contradiction in terms. You're really arguing against "rationalization".