Do you hold hands with men?
No. But I remember in Korea many years ago, there were “straight” Korean men who on some public occasions held hands with some other “straight” Korean man who we. I learned from them it had zero to with being “gay”.
It’s no different from how some Italian and French men still kiss on the cheek other male friends when they are greeting.
In both the Asian men hand holding and the kiss on the cheek of French or Italian men, yes it is a form of “affection” but it is not affection of a sexual kind.
“We” turn it into something “sexual” because it is not a usual custom among us.