For instance, Darwin is quoted from a letter to Hooker in 1871 as saying:Tangential, non-specific and tertiary;...
And Thomas Huxley, an "evangelical" atheist considered himself to be Darwin's bulldog.
And then there's Erasmus Darwin, Darwin's grandfather, who is considered to have inspired the Frankenstein story.
And of course his cousin, Sir Francis Galton is considered to be the father of eugenics, a great evil of the last century.
So there exists quite a bit of source material for anyone to fill in the gaps due to the "absence of evidence."
I see
1) guilt by association (implying that Darwin is bad b/c of his associations)
2) Hasty generalization (by making an unjustified general inference from an insufficient sample of works writing or correspondence).
3) Spotlighting - infering that highly conspicuous individuals are define the group.
4) appeal to ignorance (argumentum ad ignorantium) Proposing that something must be true if not proved false, or false if not proved true.
Can't you provide something more concrete?