I think the key difference is whether or not a man rejects hugs only from men or from all people. I’ve met plenty of people who aren’t huggers - male and female - who don’t like being hugged by anyone except maybe intimate partners or close family members, but if a man is only afraid of getting hugs from men then that’s a separate issue from general bodily autonomy.