This claim is falls in the face of historical records. Until the time period after the Nate Turner murders, MANY mixed race people, including free people of color, mulattos, octoroons and quadroons lived, owned property and worked in Albemarle County.
Court records show that they weren't afraid to take even wealthy planters to court if they felt they had been wronged.
Free people of color even owned land abutting the Dabney/Carr land.
You seem to have a rather idealized view of the status of freed slaves, mulattoes and indians in Virginia. I can assure you that it was not entirely pleasant. Better than slavery? I’d say so, but owners of liberated slaves were still obligated by law to support them under certain circumstances, such as youth, advanced age, disability, etc. so can one honestly claim that they were actually free in a legal sense, if their owners were still obligated?
Black Laws Of Virginia
http://www.racetimeplace.com/ugrr/slavelawsofvirginiasince1705.htm