2
Many existing Fediverse services are being operated by people who are opposed indexing the content on their instance(s). You may run into resistance from that angle.
I mean unless they make their instance private I don't see why you wouldn't index them? That's literally why google provided such a value in their early days.
Even Google doesn't index webpages that include "noindex" in a header. You are going to run into a lot of people who don't agree with what you are trying to do. If you start reaching out to the people running Fediverse services to let them know that you're trying to index the data on their services, you can learn what they think of the idea.
this post was submitted on 01 Jul 2023
2 points (100.0% liked)
Lemmy.world Support
3191 readers
1 users here now
Lemmy.world Support
Welcome to the official Lemmy.world Support community! Post your issues or questions about Lemmy.world here.
This community is for issues related to the Lemmy World instance only. For Lemmy software requests or bug reports, please go to the Lemmy github page.
This community is subject to the rules defined here for lemmy.world.
To open a support ticket
You can also DM https://lemmy.world/u/lwreport or email report@lemmy.world (PGP Supported) if you need to reach our directly to the admin team.
Follow us for server news ๐
Outages ๐ฅ
https://status.lemmy.world
founded 1 year ago
MODERATORS