I don't think sugar sand is entirely composed of silica... most of it is ground up calcium since it's made of coral, not minerals from the mountains like say, the sand in Virginia. You'd need to dissolve it in some acid to remove the calcium and that may still leave other minerals.
If it were just blowing around on a beach there would be pollen too.