The quality and the traffic. At least in terms of engagement. I knew another mod there that I used to do spamhunting with and we both modded a couple big subs, we were talking about it one day and we were talking about sub traffic, and I noted about 2 years ago there was actually a big decline in traffic in /r/videos, which he modded he said he hadn't noticed it, but when you went to archive.org and compared random front pages to engagement at the time, you noticed that all posts overall had fewer comments and fewer upvotes, we started checking a few more large subs and noticed it was quite similar.

Quality is, to some extent, a mod failing. Mods can't be expected to go out there and produce top quality posts all the time, but they can be expected to keep out the low quality content, and a lot of them don't do that. By ignoring frequently reposted topics, to not bothering to properly apply the rules to keep the posts fully on topic, the subs just declined and declined.