It means that the rights of black citizens was hardly his foremost concern. It, therefore, was not motivating his war effort.
And if that is grounds for condemning Lincoln then your opinion of the rebel leadership must be even worse.