Slaves could have been used for mining and railroad work. Slavers would have adapted.
Railroads maybe, but I suspect white miners would not cooperate. People have no idea how incredibly prejudiced people were back then.
But this is still a small number compared to slaves used in agriculture.