I think the structure of these apps in terms of the social interactions they encourage has a ton to do with the quality of the experience.

It's fairly clear to me that the quality of the conversation on Mastodon is simply better. I think it's because the not-quite-4chan-but-almost level of anonymity and randomness to any given interaction on Lemmy makes it easy and consequence-free to be a dick. Enabling mods to deal with it once the dickishness reaches a certain threshold doesn't really seem to do anything to solve the problem, partly because there's a huge range of dickish behavior that doesn't quite rise to the level where it needs mod attention, but still degrades the conversation and dilutes anything good anyone's trying to add.

I don't really have much experience outside the Lemmy and Mastodon communities, but I think app authors spending time on particular UX features would probably be better spending their time on studying in depth the types of interactions their apps are enabling and encouraging, and trying to keep an eye on that as they're designing the system. What those answers are I don't really know, but it seems important.