Lemmy’s continued lack of basic moderation tools with zero indication that is going to change anytime soon is the biggest problem by far. It is also still a very opaque process to join and participate for the average user and standing up your own instance is no small feat, complete with legal minefields that many don’t even realize they are walking in to. No matter how you were engaging with the fediverse, there are huge hurdles. Even being a lurker is difficult.
I am not knocking the people who dedicate their time to making it work. Every second they give is voluntary and a service for us. But it doesn’t change the fact that we just don’t have what we need. Mastodon is the only side of the fediverse that is ready for prime time, and that’s borderline/has a lot of caveats.