ADMIN, isn't it time to move from lemmy.world?
They said, from their lemmy.world account.
this post was submitted on 12 Aug 2023
-17 points (36.5% liked)
Selfhosted
40006 readers
786 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 1 year ago
MODERATORS
just a user of an instance saying things about that instance - I would find the opposite weird.
What's wrong with that? I've started here some months ago and I mainly follow this community.
The problem is it's the largest and is being attacked. Use a different instance and you'll have no problem accessing this community. If it weren't for people posting about the outages, I'd never even know.
This community is on lemmy.world so yes, there would be problems
I honestly haven't noticed. For me to see the posts they simply need to get pushed, which is pretty lightweight. And if Lemmy.world crashes after and stays down for a few hours it won't impact my ability to view the post. For a lemmy.world user the instance needs to be up for them to view any content.
In a way yes, when I post a comment, no one else will be able to see it while the site is down. But it eventually will go through when the site is up. And because I'm browsing from a different instance, I can see all the content even if lemmy.world is offline.
The problem is DDoS attacks. See: https://lemmy.world/post/2923697
But isn't Cloudflare supposed to block those attacks?
They explained that the attacks are in the form of requests that tax the database server, not the website itself.
Yep Cloudflare protects against classic DDoS (like many clients doing a lot of small requests). Here attacks are performed presumibly by users that know very well how the Lemmy's backend works and where bottlenecks are, so that with a small number of well made requests they are able to mess up the backend and Cloudflare doesn't notice it
Cloudflare DDos protection isn't a silver bullet; the attacks are distributed and come from shifting source IPs, and are sophisticated in that they exploit resource intensive queries specifically designed to overload a Lemmy instance. If lemmy.world were to pivot to some other instance, who's to say the culprits wouldn't just resume their efforts pointed at the new location? There are theories these may be carried out by the recently-defederated fringe hate communities
From what I understand, Cloudflare can block some DDoS attacks, but not all of them.
The attacks on Lemmy have to do with poorly optimized SQL requests; these are requests that shouldn't take long to execute, but do due to some oversight. By spamming these requests, the attackers can bring Lemmy on it's knees.
Actually, wouldn't this attack better be categorized as a DoS attack ? What's so distributed about it ?
Cloudflare has DDoS protection but it can't stop everything 100% of the time. According to the admins, the attackers are very familiar with how lemmy works and are using this knowledge to overwhelm resources. This isn't just a simple script kiddy or bonnet for hire but likely points to someone that has worked within the lemmy community.
https://www.cloudflare.com/learning/ddos/what-is-a-ddos-attack/
https://www.cloudflare.com/learning/ddos/glossary/web-application-firewall-waf/
It can help, but is not perfect. There's kind of been an arms race between services like Cloudflare and script kiddies/hackers DDOSing: Their methods became a little more sophisticated to the point that they keep the traffic shifting from address to address so it's harder to track/block.
They'll do other things to mess with the host too like spamming "white noise" pictures to fill up server storage space, so while DDOS attacks play a role, there are other issues at play on top of that.
For what it's worth, I've been using Kbin.social and sh.itjust.works as well and they have also had some issues here and there. No host will be perfect and invulnerable from every attack, and when an instance becomes more popular, it becomes increasingly likely to be targeted by attackers.
Ironic. They could self-host for others, but they couldn't self-host themselves.
There's a difference between self hosting and running a public service. There's a lot more overhead running a pubic service.
What's wrong with that? I've started here some months ago and I mainly follow this community and I don't blame anyone.
I find this comment section a prime example of dickish hivemind seething over nothing.
There's a dude, obviously quite fresh in the ways of Mastodon. He probably doesn't realize all the nuts & bolts supporting the system and how it all works. He is asking a question that is logical, but it needs clarification, like "it doesn't work like this, my man".
Instead he gets "Hsssssssssssssss, selfhost it, hsssssssssssssssssssss, interloper, hsssssssssssssssssss, you want to destroy this place, hssssssssssssssss..."
Get a life, eejits.
Your post is more offensive then the others.
Not to mention that the OP is open for interpretation, and it came across (at least to me) as another entitled person moaning about a free service.
Next to that, teaching people about the selfhost option is actually what lemmy is all about. Not beeing owned by a single corporate entity, so endusers have freedom to choose.
If lemmy want to survive, it depends on people who know how to selfhost it.
They are making an assertion as if it is a statement of fact...and they are, in fact, wrong. That's ignorant and not helpful for discussion or helpful for understanfing and solving the actual issue. If they had actually asked an innocent question it would be different, but they didn't. That is why the responses are the way they are.
No. The responses are the way they are, because people who gave them are already thinking they joined some elitistic "muh sikret klub!" group.
Simple "eh, it won't fix the problem, and here's why and how YOU can help" would be preferable, but no, special elite force of lemmy underground is too privileged to bother.
Thank heavens not everyone is like that. Saves the number of times I have to hit "block the idiot" button.
Thanks! As you, I thought that this was a community to share thought and knowledge. In fact a couple of guys here pointed out that I could solve my problem accessing this community in another instance, not in the better way, but at least that was helpful!
Yeah. It's good that there are users who actually take time to explain some stuff, rather than just hissssssss like rabid vipers merely because somebody - oh no, what a preposterous idea! - asked a harmless question.
Well your account is on lemmy.world so how d'ya know the issue isn't with your own access to the front end?
Many don't interact with the lemmy.world directly, so we might only see delays in post propogation (if there is such an issue on the backend - I don't see any but could be wrong).
I agree picking the biggest instances isn't great from a scaling perspective, but s'gonna be hard to move any community once established.
The problem is not the server. One of the reasons is because lemmy world is being DDoS attacked by certain party with grudges. Because lemmy is not used to this kind of attack, which is made difficult by way it is designed and open source nature where the attacker can easily uncover vulnerabilities, it took some times and learning to mitigate them. You can open new account in other instances or move if that's too much inconvenience for you. No one is stopping you.
By claiming that the problem isn't DDOS, you're just advertising your ignorance. Cloudflair is outstanding for protecting static web content against DDOS, and Lemmy.world is well protected against that. The problem is certain dynamic pages and api calls that can only be rendered from costly realtime dynamic database operations...those are the url that the DDOS attackers are focusing on and those are the kinds of content that cannot be easily protected by cloudflair.
Your premise, though, is still accidently correct. The way to mitigate instances being targeted by DDOS is to spread the user base and community hosting across a vast number of instances so that no one instance is such a rewarding target for DDOS attack.
You're right about my ignorance about Lemmy, I'm a user on this federated ...thing and I know nothing ahout the Lemmy server. Being in a selfhosted community your answer is what I was expecting (maybe with less attack). The API are used only by the federated instance or also by the smartphone apps? For what I see, it seems to be the former, and, if it really is so, the API calls could be allowed only by those server and blocked from everyone else, Cloidflare WAF can do this. I know that the servers are a lot, but it could be possible to insert in the WAF all the IPs of the federated instances. ...or not?
What makes you think other servers aren't having similar problems?
(spoiler alert: they all are, lemmy is an immature platform)
Is it possible to migrate a community from one instance to another?
No, not programmatically. The only way to 'move' an instance is for a community of users to create a new community elsewhere and arbitrarily start using it as their preferred community. There is no way to force it and there is no mechanic for moving.
I think the admin of c/selfhosted is the admin of Lemmy.world
Have you donated money so they can?
Another server (and i assume you mean one with better specs) means more money. Its not just challenging, its also costly.
Alternative you could setup your own lemmy instance, you can still join the community’s on lemmy.world or lemmy.ml or whereever :) and its fun to do.