Those of us who remain Jews are direct descendants. Some, it may surprise you, have the genealogy record going back thousands of years.
Traditionally, Jewish culture has essentially been exclusionary by nature. Thus, you may be correct in that the past mixtures have blended into the surrounding populations, while the "purer" groups have continued with Jewish culture.
This particular strategy may not be working any more. In order to maintain any Jewish population base in the US, the Jewish community must develop mechanisms to reach out and include the mixed children. As I mentioned, these mixed kids now make up the majority of Jews' children. Without them, there simply won't be enough Jews to maintain any sort of population base or culture in America.