I guess the problem is that, traditionally, closer wasn't really something you made a career out of - it's where you go when you've got some good stuff but can't hack it as a starter any more. Dennis Eckersley spent 11 years as a starter before heading for the bullpen. Hell, look at Jon Smoltz, for that matter. And DH is pretty much the same way - DH is where you stick the guy who can still swing the bat, but he's got a gimpy knee or he's too fat or too old or just too slow to play the field. Now, with relievers, that's changing some, and you see more and more guys like Rivera who really do make careers out of being a closer, but DH is going to carry the feel of "dumping ground for broken fielders" for some time, I think, which is going to continue to be a hindrance to players who have their eyes on the Hall.
Before the advent of the DH, that position was often referred to as first base. Many a prolific hitter was shifted over to 1B at the tail-end of their careers, where they persisted as a below average fielder that hurt the team defensively. I agree with your assessment that this perception of the DH position will be a hindrance to making the HOF. I'm just not convinced it's a fair one.