Whitman was pretty odd, and quite scandalous in his day, but as C.S. Lewis says, the moderns interpret every expression of affection or honest love between males as "he's gay!" ("What, Boswell and Johnson (a pretty flagrantly heterosexual couple), and all those hard-bitten hairy old toughs of Romans in Tacitus asking for last kisses when the legion was broken up . . . ALL pansies? If you can believe that, you can believe anything.")
Whitman was celebratory of affection between males and the human form, including the males, but I haven't seen any information that he ever acted on it sexually . . . we just have to quit buying into the propaganda.
. . . and, btw, it might or it might not affect the quality of his poetry. THESE days, it seems that a writer's political, social, and sexual leanings positively INFECT his work.
MWith Whitman there is very little counter-argument though. There aren't a lot of scholars these days who argue that he didn't harbor romantic attractions to other men.