Support / questions about Lemmy.
Are you running lemmy 0.18.0? It breaks a bunch of internal network connectivity. The short term solution is to add an external network to both the lemmy-ui and pictrs containers, then you can change http://pictrs:8080 to http://your_domain.
Note: there are also some issues in 0.18.0 with comment federation at the moment. Please reply to confirm this comment was seen, thanks!
I can see your comment on my instance running 0.18.0!
Yay! Small victories
I've updated to 0.18.0 as well and can see your comment
Yay! Thank you for confirming
Well, butter my butt and call me a biscuit. In implementing this, I stumbled upon an issue: I had the url in pictrs_config, when it should have been pictrs.
Now at least the logs are showing an error that matches the toaster:
2023-06-25T13:09:27.753749368Z 2023-06-25T13:09:27.753609Z WARN lemmy_server::root_span_builder: error decoding response body: expected value at line 1 column 1
2023-06-25T13:09:27.753809022Z reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }
I already have those containers in an external network, and changing the URL hasn't fixed it.
Oh, this span builder error. Yeah I’ve been seeing that error since I upgraded to 0.18.0. Sorry, I don’t have a solution for this one.
Glad you figured out your pictrs error though! One other thing I did was update my resolve.conf so that valid DNS hosts were making it to the containers. Also try rebooting your host.
Feel free to add to this discussion. https://github.com/LemmyNet/lemmy/issues/3314
One other thing I did was update my resolve.conf so that valid DNS IPs were making it to the containers. Otherwise the containers might not have valid DNS. Also try rebooting your host.
Here is what I did to fix the container DNS lookups failing:
https://stackoverflow.com/questions/20430371/my-docker-container-has-no-internet
Yeah no, DNS works in the containers:
dot@deepthought:~# docker exec -it lemmy-lemmy-1 ping pictrs
PING pictrs (192.168.7.2): 56 data bytes
64 bytes from 192.168.7.2: seq=0 ttl=64 time=0.079 ms
However, your hunch did actually turn out to be correct in that there's still a networking issue: I opened a port for pictrs and hit the API with postman, and noticed it would show up in the pictrs logs which were empty before.
So I changed the url to the host's IP and the open port, and now it works. :)
lemmy.hjson:
pictrs: {
url: "http://192.168.123.21:4808"
api_key: "apikey"
}
docker-compose.yaml:
pictrs:
...
ports:
- "4808:8080"
Oh, good idea! Whatever works for you. I spent several hours yesterday trying all sorts of networking hacks to resolve the issue on my instance. I eventually found a combination that worked for me.
The concern here is we are all solving this issue in slightly different ways on our self hosted instances. Eventually, I hope the lemmy dev team releases 0.18.1 fixing all these issues for good.
Did you run the chown -R command for the volume bind? Is the pictrs actually up and running? Check docker ps | grep pictrs and see if it is actually running. Also check the logs to see if there are any errors.
I did.
dot@deepthought:~# ls -al /sharedfolders/apps/lemmy/volumes/pictrs/
total 12
drwxrwsrwx 3 991 991 4096 Jun 22 20:24 .
drwxrwsrwx 5 root root 4096 Jun 25 09:34 ..
drwxr-sr-x 3 991 991 4096 Jun 22 20:24 sled-repo
dot@deepthought:~# docker ps | grep pictrs
14bd3db41d50 asonix/pictrs:0.4.0-beta.19 "/sbin/tini -- /usr/…" 4 hours ago Up 4 hours 6669/tcp, 8080/tcp lemmy-pictrs-1
dot@deepthought:~# docker logs lemmy-pictrs-1
2023-06-25T07:34:38.379801Z INFO actix_server::builder: starting 6 workers
2023-06-25T07:34:38.379855Z INFO actix_server::server: Actix runtime found; starting in Actix runtime
Hm... What else... Did you modify the docker-compose.yml by chance? Are lemmy and pictrs sharing the same network? Other than that I'd be at a loss.