Before the advent of the DH, that position was often referred to as first base. Many a prolific hitter was shifted over to 1B at the tail-end of their careers, where they persisted as a below average fielder that hurt the team defensively. I agree with your assessment that this perception of the DH position will be a hindrance to making the HOF. I'm just not convinced it's a fair one.
Well, yeah, and when the DH has also been around for eighty years or so, that stigma may be somewhat less too ;)