I think lemmy is too small at the moment to downvote in the same way people did on reddit. It's still growing and we need people to post content to build the site and everyone's engagement, downvotes for just disagreeing are going to discourage a whole new group of people who didn't post on reddit because of getting drowned out in the noise. Look in all the question posts about how people are enjoying lemmy and you'll see a bunch of them.
I've just been moving on and ignoring things I disagree with, no upvote is enough punishment in my eyes. Saying that I've been giving out upvotes to most things to encourage people to post and comment more. If something is clearly gross and offensive it'll get a downvote and possibly a report, but that's so rare in the communities I'm going to. If it's a really off topic comment or whatever then yes it's downvoted because it doesn't belong.
If there's something I really, really disagree with but don't want to get into a conversation about it I'll just block the user. That way I don't have to see or deal with their opinions I disagree with anymore and can just move on with my life.
I guess that's a really, really long winded way of agreeing with you lol. I've been a lot more liberal with the upvotes and have really changed my downvote behaviour too.