Nope. For example endogenous retroviruses (ERVs) can be identified by the fact that they have the genes for reverse transcriptase. This does not depend on any assumptions about phylogeny.
The observed fact that ERVs are found in the same place in the genomes of different species is also independent of any phylogenetic assumptions.
It's also a fact that if species are classified by whether they contain various ERVs or not then they always form a tree structure; again, this is independent of any phylogeny.
It's the last fact, that the ERV-derived tree always matches the phylogenetic tree that was constructed using non-genetic data (anatomy, behavior, geographic distribution, etc) that makes ERVs (and other genetic markers like pseudogenes) so interesting.
To sum up: by using nothing but gene sequencing, living organisms are naturally organized into a tree structure. This tree just happens to be the same as the phylogenetic tree already constructed by techniques that don't use genetics. Common descent is the simplest and obvious explanation for these facts.
If you assume that common descent is true *and* that ERV 'insertions' actually occurred *and* were a singular event that occurred once in the history of a 'common ancestor', then they mean something.
All of the thousands of assumed human ERVs are defective and produce no virus particles. Human ERV 'insertions' are 'assumed' events, not observed.
It seems to me that the assumption that ERV infections occurred once and only once and can be 'traced' is a big weakness. You would need to show proof that ERV infections can only occur one in a population genome and then move to fixation in each species in which they occur. That's absurd.
If you think that you do not use 'any phylogenetic assumptions', you would be wrong.