I don't think this reference is to a black woman. The Shulamite was almost certainly an Israelite girl.
But she worked in the fields and vineyards and was therefore tanned. In most societies throughout history, beauty is associated with wealth.
Wealthy women don't have to work outside and don't get tanned, so paleness in these societies is generally associated with beauty.
BTW, this all changed when most people began working in offices or factories. Being tanned meant you could afford to be outside lounging on the beach or playing tennis, so IT became a sign of wealth and privilege and therefore of beauty.
Paleness is also apparently a sign of beauty in S Korea judging from all the references I see in Korean television shows.