First, votes aren't exactly transparent, but they also aren't completely private either. User voting records are stored in databases that instance owners have access to, so it's possible for them to see (and/or even publish) up/down voting history. KBin already does this publicly. So I can see an argument being made that if the info is available to some people, it should be available to all people.

Personally, I wouldn't care if my upvotes and downvotes are exposed to the commenters/posters that I voted on, but I'm concerned about the possibility of it being used for discrimination. Imagine me following/participating in a community and then being immediately banned from that community solely because a community moderator didn't like how I upvoted/downvoted on things. For example, say I want to participate in a philosophy or politic themed community and one of the mods there just happens to be very conservative and decides to exclude me just because I upvoted something that was NSFW once upon a time and they disapproved of that behavior? This will absolutely happen if all voting is public. On reddit, a similar form of discrimination happened by analyzing where people posted and they would be banned from certain subreddits just based on the other subreddits they have been active on- and even worse was that this was often done by a bot without regard for the actual comments made. I recall a very specific example of someone who used to hop into r/conservative to challenge or antagonize certain lines of thinking and they were banned from liberal/progressive subreddits because of their activity on r/conservative despite the fact that they were not sympathetic to anything on r/conservative. That same discrimination can (and probably does) happen on Lemmy already, but making voting history public will take it to the next level.

If voting ever did become public on lemmy, then at a minimum users should be able to see/review/audit their voting history and be given the ability to retroactively delete some/all of it.

You're also ignoring the fact that it's trivial to create/use alternate lemmy accounts. If voting records were public, it would just drive people to create multiple accounts from which to vote on things - to compartmentalize their interactions with different communities or users. Since this fact means that users would STILL be able to hide/mask their voting history, I think this is a good argument that it makes no logical sense to make voting records public.

I think an ideal solution would be for users to just have a choice to make their voting public or to keep it private, or to selectively publicize or keep secret on a vote-by-vote bases.

I forgot Lemmy users can't see who upvoted/downvoted something (votes are still visible in Kbin)

I've noticed there are people who just spam with downvotes for no reason (downvoting a certain community's posts, downvoting all comments under a post, downvoting all posts/comments made by someone they may not like etc), it's pretty weird