Well, this is a bit uncomfortable. I wasn't aware that votes were shared as well; IIRC an admin said that the occasional downvote brigades we've had couldn't be reversed without losing posts, so I assumed the votes were anonymized somehow 😐
I wonder how feasible it would be to only send an anonymous vote count for every post/comment to other instances, keeping the details within the instance's database only. Federation is already "broken" when an older database has to be restored or even when someone is banned from an instance, so it doesn't seem to be fundamentally necessary for all instances to match
More importantly, is more sensitive data also shared? I would hope that IP addresses, which posts you've viewed, etc. aren't stored anywhere, or at least not forwarded to other instances