It's definitely a thing that happens. I notice sometimes that I'll get 1 downvote across all my recent stuff, even the stuff that's super anodyne, like posting a link to a historical artifact when someone asks for it. Apparently, a space in their head is rent-free, and you're occupying it.