Mulattoes and the offspring of mixed race couples and couplings were referred to by various terms, depending upon their skin color.
IIRC, High Yellow was one almost white, but less than swarthy...
There are other terms but this one came to mind.
What is swarthy?
So yellow is mixed race?
Must be regional expressions?