That means, first, that any mutations altering melanin content would have a hard time getting established, so they'd have to re-occur dozens of times for a reasonable chance at fixation. Second, it would be a near-neutral mutation, such that natural selection would take 'forever' to fix it in any population of a reasonable size.
That said, I fully expect (predict) that we'll eventually learn that the diversity in skin color has nothing to do with random mutations, but is expressed from latent genetic information (i.e., pre-adaptationism).
The problem is that humans didn't evolve by natural selection, but by tribal warfare, which is a very unusual high-speed form of evolution that modern academia is loathe to admit. Most human traits, such as having free lubricated arms to wield weapons with, religion, intelligence, communication, culture, morals, altruism, bravery, can all be traced to their advantage in war. And shockingly, most cosmetic preferences were made by mothers, favoring for example babies with blond hair and blue eyes over siblings without. It's hard for us to understand in modern times but mothers had more mouths to feed and significantly less reliable food sources, so often had to choose which children lived. Some cosmetic traits were chosen by men, such as big boobs and blond/blue surviving into adulthood, but the original blond hair/blue trait was cultivated by mothers.