this post was submitted on 05 Jul 2023
25 points (85.7% liked)

Selfhosted

40219 readers
926 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

The 6 nines mean that an ideal service should have 99,9999% uptime, right?

That's almost 32 seconds of downtime in a year!

If so, how much would it cost to do it? (Let's consider that is a marketplace site with 1000 daily users)

top 11 comments
sorted by: hot top controversial new old
[–] sijt@lemmy.world 28 points 1 year ago (2 children)

It’s really hard. And really expensive. I used to work in five nine environments, life or death type use cases, and my rule of thumb was that you double your cost for every extra nine you add.

When we got to five nines it was multiple hot standbys with a custom control and orchestration plane - literally custom hardware we had to build. This was for local installations, so not modern cloud environments (it was over a decade ago), but many of the challenges are similar, like session handling, transmission replay and caching, locking, clashing, routing, jitter, latency etc.

Looks like it is very stressful to work when you need this amount of availability, even more with the pressure that a little error can cause giant consequences.

Thanks for your answer!

[–] boonhet@lemm.ee 2 points 1 year ago

I'm sorta surprised that it's only 2x per nine. Six or seven nines sounds ridiculous.

[–] iAmNotorious@lemmy.world 17 points 1 year ago* (last edited 1 year ago) (1 children)

6 nines is really really difficult. It’s hard to estimate costs without specific requirements, but a marketplace site with 1000 daily users means you’re expecting about 1 user per minute, which isn’t a lot. I’d imagine you could get by with the cheapest cloud hosting.

The real problem is that most major cloud providers don’t offer 6 nines. Even AWS only offers credits below 99.5%, so you’d want to not lock yourself into a single provider. My best suggestion is to have a small/cheap server with all of the big names and load balance/round robin between them.

[–] SpacePirate@lemmy.ml 12 points 1 year ago* (last edited 1 year ago)

This.

At some point, you need to be able to quantify the risk to your business before you can do this.

For instance, if your business earns $10 per transaction, and you perform 100 transactions per second, the difference between five and six nines (313 seconds vs 31 seconds) is $282,000; nowhere near enough to justify the added investment.

Edit: Important to note that for the first example, these are already enormously huge numbers. Such a business, assuming no holidays or weekends, would be grossing $31.5 billion per year, in the same ballpark as Oracle and Coca Cola.

So when we say the company is losing 282,000, this is a tiny, tiny fraction of revenue. Even 99.5%, which is almost two days of downtime, would “only” be a loss of 0.5% of all revenue for the year. Sure, this is $157M, but even that would probably not cover the cost of a six nines infrastructure (that said, they could save up to $120M per year by achieving 99.9%, which would be worth exploring).

[–] Psiczar@aussie.zone 13 points 1 year ago (1 children)

This is such an open ended question that nobody can give you an accurate answer as there are so many factors that need to be considered.

How big is the site? How much data is being stored? What is the DB backend? How does it handle failover between DB servers in the event the primary goes down? Is it being hosted in a cloud service, your own DC or a cupboard? Does the location it is being hosted already have redundant power and internet connectivity? Are they diverse, so if one provider fails the other one will remain online? You’d need to maintain separate sites in case one location goes down due to a major event like an earthquake, so you need to replicate data in real-time to your DR location.

There are so many factors to consider and I haven’t named them all. Regardless of what your answer is, it would be very expensive to maintain any server at 6 9’s level of availability. For a marketplace website with only 1000 visitors a day there is no need for that level of availability because there would be times in the day that nobody would be on it. Only a marketplace the size of Amazon would consider that level of availability.

Whoa, there are a lot more factors than i was thinking.

I am not planning to host a site, I was just thinking about it and how complex it would be to make such website and well, makes sense to only the biggest sites on the web have that level of availability.

Thanks for your answer and for your time!

[–] SheeEttin@lemmy.world 10 points 1 year ago (1 children)

Sure. It just depends on how flexible your definition of "downtime" is.

[–] otl@lemmy.sdf.org 2 points 1 year ago

Exactly. I think about an email hosting service. What does downtime mean exactly? Is the service down when you can send but cannot receive mail? Or when the service receives mail but it can't deliver it to your inbox? Nailing this down can be difficult, especially (in my experience) with the ones who decide where $$$ goes.

[–] Osayidan@social.vmdk.ca 4 points 1 year ago (1 children)

Possible yes. Cost effective / valid business case probably not. Every extra 9 is diminishing returns: it'll cost you exponentially more than the previous 9 and money saved from potential downtime is reduced. Like you said 32 seconds of downtime, how much money is that for the business?

You're pretty much looking at multiple geographically diverse T4 datacenters with N+2 or even N+3 redundancy all the way up and down the stack, while also implementing diversity wherever possible so no single vendor of anything can cause you to not be operational.

Even with all that though, you'll eventually get wrecked by DNS somewhere somehow, because it's always DNS.

It's always DNS (bad memories showing up)

load more comments
view more: next ›