Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.

I am still trying a few things, but I am not very optimistic that I can get access.

The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.

As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable 😓

I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing 😥

Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.

(page 2) 50 comments

sorted by: hot top controversial new old

[–] Blaze@lemmy.dbzer0.com 21 points 2 days ago

Thank you for letting us know, good luck!

[–] FrostyTrichs@crazypeople.online 17 points 2 days ago

Ouch. Best of luck getting it back up and running ASAP.

[–] fxomt@lemmy.dbzer0.com 11 points 2 days ago (1 children)

Good luck and all the best. Time to touch grass, everyone 😔

[–] psychepomp@lemmy.dbzer0.com 5 points 1 day ago

You mean create an alt, right?

[–] MelodiousFunk@startrek.website 13 points 2 days ago

Thanks for the heads up, try not to stress too much.

[–] AlexisFR@jlai.lu 9 points 2 days ago (2 children)

That's unlucky, but is it hosted at someone's home?

load more comments (2 replies)

[–] perestroika@lemm.ee 8 points 2 days ago* (last edited 2 days ago)

Thanks for the info, a healthy amount of real life awaits then. :)

I knew it was running on solar energy and old hardware so I guessed something like this had happened.

If you need fail-over to awaken a backup system when the primary fails, things can be designed. :)

[–] match@pawb.social 7 points 2 days ago

Ty for the update! slrpnk is my favorite instance that isn't my home instance

[–] bacon_pdp@lemmy.world 10 points 2 days ago

It happens, it’ll be fine. Just remember not to stress out too hard about it and it’ll be fine. The solar punk community knows how to survive a server outage in style.

[–] khaleer@sopuli.xyz 8 points 2 days ago* (last edited 2 days ago)

It looks like there is no enough solar power for solarpunk!

~~no need to show me where doors are, I will find them by myself~~

anyway, good luck with that!

[–] tenchiken@lemmy.dbzer0.com 6 points 2 days ago

Sending positive vibes, and passing offer of technical assistance once you get access again.

Hopefully it's an easy fix once you get into things again.

[–] PM_ME_VINTAGE_30S@lemmy.sdf.org 4 points 2 days ago

That sucks. Thank you for your time and running the instance. Gonna miss you all like hell while you're gone. 😞

[–] Brkdncr@lemmy.world 1 points 1 day ago

Opengear cellular backup appliance with serial/console connectivity to hardware.

[–] originalucifer@moist.catsweat.com 4 points 2 days ago (1 children)

people hate the big cloud solutions, but this is the kind of thing their HA infrastructure prevents against... hardware failures

i dont enjoy using (or paying for) aws, but i will never have a firewall or disk failure.

[–] drspod@lemmy.ml 7 points 2 days ago

Their HA infrastructure is all built on open source projects. The thing they have that we don't is teams of SREs on-call 24/7.

load more comments