Measuring existing relics in 1870 in no way accounts for all the pieces bought and sold over the centuries; many such pieces could have been destroyed, lost, discarded as fakes etc. over the years
Even if ten times as many pieces had been lost, burnt by Saracens etc - that takes the tally to 1/5 of a cross.
That's ... a pretty small forest. And that's all that the historical record actually tells us. Anything else is unfounded conjecture.