The source is the 1860 census, as summarized here.
In the case of Virginia, it says the state's white population lived in 201,523 families, of whom 52,128 or 26% owned slaves.
The chart shows numbers for other slave-holding states, which range from 3% of white families owning slaves in Delaware to 49% in Mississippi.
24% is an amazing figure. It is just hard to believe. Not that you didn;t provide a source, just that, wow, 1 in 4.