I have to agree --- I suspect Downs would agree --- that statistics are hard to get with any precision, especially at a time when a census esp. of black slaves and freedmen was hardly exact. However, slaves appeared in property lists (e.g. estates, bequests) and they were enumerated decenially by the US. govt. in the antebellum days for apportionment purposes (House districts.)
So realistic estimates could be made.
Statistical extrapolations only, that's what we have.
People look at the population growth rate from 1850 to 1860 and say, "if that same rate of growth had continued until 1870, then populations in 1870 would have been XX-thousands more than they were."
The difference between projected and actual they call "civilian deaths caused by war".
But what are we really talking about?
However, actual data to support any of these factors is entirely lacking.