RoundSparrow

joined 1 year ago
[–] RoundSparrow@lemmy.ml 2 points 1 year ago* (last edited 1 year ago) (6 children)

Lemmy is pretty immature as code to actually run in production. It may be well over 4 years old, but the whole thing seems to have very little in the way of information that a server operator can look at to check the health and problems under the covers. It also doesn't deal with unrecognized data very well and hides a lot of errors in a log where the messages are often not very much of a hint what is going on.

Lemmy surely is unique, as I almost never see people using it actually criticize the code for quality assurance and testing. More often than not, I see people cheering and defending it. I've had to look through this experience and code as it is more run like an art project or a music band than any serious focus on data integrity or performance concern.

[–] RoundSparrow@lemmy.ml 2 points 1 year ago (4 children)

if it were me right now with Lemmy 0.18.4, I'd take the server offline, do a PostgreSQL dump file - keep a copy, then hand-edit the sequence numbers in the dump file - and do a restore.

you probably only had a few users, so I would set user to 100, person id can be higher because of federation - but jump ahead to 10000 maybe. Post and comment set ahead to 10000 ... and community set ahead to 10000 because that gets federated

the PostgreSQL sequence numbers should only get used on newly created objects here-forward.

[–] RoundSparrow@lemmy.ml 1 points 1 year ago (3 children)

Right now lemmy backend only supports a single 'subscribed' list, that would be like a music player only having a single playlist. I know people are using multiple logins just to have more flexibility in managing their lists.

[–] RoundSparrow@lemmy.ml 1 points 1 year ago (6 children)

GitHub issue about this topic: https://github.com/LemmyNet/lemmy/issues/3782

Some of it might be avoided by tweaking the PostgreSQL database with higher key values after the new install, but the whole situation isn't really planned for or recognized in the code

[–] RoundSparrow@lemmy.ml 1 points 1 year ago (8 children)

You really have blind faith that federation even works, when I've been validating data and highlighting that delivery is not reliable when it has so much overhead it crashes servers.

[–] RoundSparrow@lemmy.ml 14 points 1 year ago* (last edited 1 year ago) (3 children)

This basically shuts my idea down

it's not very difficult to modify the code for something like this.... and closing off registration wont' let anyone else login and create new content form your istance.

Personally the load on the major servers by having one more instance that subscribes to everything is why I think people should back off from creating more than the 1500 instances Lemmy network already has. Delivery of every single vote, comment, post 24 hours a day just so one person can read content for an hour or two a day.

That makes sense for email systems where all that content doesn't have to be sent, but for Lemmy it's a huge amount of overhead.

[–] RoundSparrow@lemmy.ml 2 points 1 year ago

ok, I'm going to delete this post. People actually aren't discussing privacy and are just debating if they think Lemmy needs Multi-Reddit. And I just want to get the code finished. I am probably moving ahead on the code with ZERO sharing of any existing data.

[–] RoundSparrow@lemmy.ml 0 points 1 year ago

dismissing client-side techniques as nonsense without seeming to understand why they were being discussed in the first place.

I'm the one who started a post about a server-side solution that entirely is based on Reddit's code for a server-side solution. YOU are the one coming along with this wild idea that a server change isn't needed at all. yet, you have not demonstrated this wild claim you made!

I’m not interested in any multireddit feature that reduces sub privacy. I’d consider it a net loss for lemmy.

It does NOT require it. I will repeat it: IT IS NOT REQUIRED! It is a sub-feature that facilities better openness that I am suggesting be added as part of the core feature I'm developing.

On Reddit, multi-reddits personal in nature.

10 years ago Reddit announced it as entirely not being personal! That sharing them was the whole point. I again question if you even understand what multi-reddit is!

[–] RoundSparrow@lemmy.ml 1 points 1 year ago

shouldn’t require relaxing privacy constraints in any case.

It isn't at all essential to the feature.

I have already coded it so that it does NOT require sharing of anyone's data, at all. No way shape or form. I'm proposing it as a discussion topic because it's easy to implement and goes along with the whole spirit of a public forum where people share their public stuff. That people might actually want an easy way to help others out...

But, it's easier for me just to avoid any privacy topic entirely and not allow sharing of anything. Just build the whole design with opt-in only empty list.

[–] RoundSparrow@lemmy.ml 2 points 1 year ago (2 children)

I’m suggesting that multireddits are a “local” function. Theu are so local that they’re possible without server-side support at all,

Again, how? If I want a blend of 50 different communities, how can Reddit or Lemmy do that without 50 API calls if you do not add server-side MultiReddit code?

50 API calls is the overhead and nonsense that is being avoided here....

[–] RoundSparrow@lemmy.ml 1 points 1 year ago

It could also be a filtered view based on the subscribed/all feed which provides a single API call that can return material from multiple communities.

"that can return material from multiple communities" - that's exactly how Reddit does multi-reddit, what feature do you think multi-reddit is?

[–] RoundSparrow@lemmy.ml 2 points 1 year ago

But it should definitely be off by default and have a clear warning when you try to enable it.

I was afraid people would say that. The easier way is to just not touch it at all, as adding new code to opt in/opt out is more Rust code programming that is in rare supply with developers.

The easiest solution is to avoid it and not introduce sharing of personal communities at all. Which was what I was afraid this discussion would yield. So we start fresh with empty MultiPass lists and build them up from scratch.]

 

If you visit a popular community like /c/memes@lemmy.ml with your web browser, the images shown are hotlinked from the Lemmy instance that the person posting the image utilized. This means that your browser makes a https request to that remote server, not your local instance, giving that server your IP address and web browser version string.

Assume that it is not difficult for someone to compile this data and build a profile of your browsing habits and patterns of image fetching - and is able to identify with high probability which comments and user account is being used on the remote instance (based on timestamp comparison).

For example, if you are a user on lemmy.ml browsing the local community memes, you see postings like these first two I see right now:

You can see that the 2nd one has a origin of pawb.social - and that thumbnail was loaded from a sever on that remote site:

https://pawb.social/pictrs/image/fc4389aa-bd4f-4406-bfd6-d97d41a3324e.webp?format=webp&thumbnail=256

Just browsing a list of memes you are giving out your IP address and browser string to dozens of Lemmy servers hosted by anonymous owner/operators.

 

At the time of this posting, both sites for over a minute were throwing 500 or 502 errors. I've seen this 'naked nginx 50x error page' many times throughout the last week from almost every Lemmy site I visit.

First off, how do we configure Nginx with a better error message when the site is down that says try again in a few minutes, etc? Someone have some config steps to share?

 

cross-posted from: https://lemmy.ml/post/1216911

Some of you may have noticed that federated actions are slow to synchronize between Lemmy instances. This is most likely because of the setting "Federation worker count" under /admin. It determines how many federation activities can be sent out at once. The default value is 64 which is enough for small or medium sized instances. But for large instances it needs to be increased.

Grep the server logs for "Maximum number of activitypub workers reached" and "Activity queue stats" to confirm that this affects you. For lemmy.ml I just changed the value to 512, you have to experiment what is sufficient. The new value is only applied after restarting Lemmy. In my case changing the value through the website didnt work (maybe because its overloaded). Instead I had to update local_site.federation_worker_count directly in the database.

Edit: I had to increase the value to 160k for lemmy.ml. Now the stats arent getting logged anymore, so Im not sure if the pending queue is still building up or not.

 

I will post comments on individual topics

 

Right now my focus is on testing the app and trying to get some understanding of how the code works.

Enabling federation makes sense because I want to be able to test and study how it interacts with other instances.

If I purge the install and reinstall, how does that impact the identity of the server to other instances?

Obviously the users, communities, comments would all get removed...

Is there some kind of encryption key that identifies an instance? Or is it the domain name alone that does this?

 

I'm still getting familiar with the code. Are these tunable parameters?

As of this moment, the !chatgpt@lemmy.ml Community isn't yet usable over on beehaw https://beehaw.org/c/chatgpt@lemmy.ml - and it's been local here on lemmy.ml for about 1 day.

I also see it takes about 1 day for a update to a user profile to be picked up on beehaw.

I do see if I edit a comment the change gets replicated to from lemmy.ml to behaw in a matter of seconds.

Is that generally what people have experienced?

 

Experience and tips based off these Ubuntu 20.04 instructions:
https://join-lemmy.org/docs/en/administration/from_scratch.html

I thought trying to detail problems and extra steps in public might help some people.

view more: ‹ prev next ›