My experience, so far, has been quite the opposite to yours, in that, I've not had an issue with accidentally openning posts while scrolling with Voyager. For me, this seems more of an issue in other apps I've tried.. That's not to say that it isn't happening for you, of course..

Is this gesture related? I have disabled gestures because it was too sensitive for me (upvoting posts, saving posts all automatically 😅).

After disabling it works very nice, I just vane to open the action menu for me to upvote or reply.