Saying a specific fossil is a “direct ancestor” is a bit like saying that since someone is of Austrian descent, Mozart must be a direct ancestor.
I think it's a little more specific than that. We're talking about species and their predecessors.