Technology

59190 readers

3487 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

Does archive.today break when using private DNS (quad9)? (discuss.tchncs.de)

submitted 11 months ago by _edge@discuss.tchncs.de to c/technology@lemmy.world

13 comments fedilink hide all child comments

Today, I enabled quad9 dns for my home network, and archive.today now requires a captcha, which results in an infinite loop.

A similar problem was reported some months ago for Cloudflare's 1.1.1.1

Posting here to see whether it's just me or everyone. Is this a know problem?

you are viewing a single comment's thread
view the rest of the comments

[–] NonDollarCurrency@monero.town 6 points 11 months ago (1 children)

Yes it's been like that forever. Before it used to outright block the entire domain.

[–] _s10e@feddit.de 1 points 11 months ago (1 children)

Who's blocking what?

Last time, IIRC someone blamed Cloudflare and they said they did not do anything, just relaying from upstream.

[–] redcalcium@lemmy.institute 6 points 11 months ago (3 children)

The gist is, archive.today configured their DNS server to use edns client subnet to determine the visitor's general location to direct them to servers closest to their area for load balancing purpose. Cloudflare DNS however doesn't pass that information for privacy reason. I guess this piss archive.today's dev off because their dns-based load balancing is no longer work effectively for cloudflare DNS users, so they outright block it.

[–] _edge@discuss.tchncs.de 2 points 11 months ago (1 children)

Weird, I change from dns11.quad9.net (with ECS / EDNS client subnet enabled) to dns.quad9.net. Now archive.today works.

[–] _edge@discuss.tchncs.de 1 points 11 months ago (1 children)

And then it broke again. And then it worked again.

Totally random. How does one debug this?

[–] redcalcium@lemmy.institute 3 points 11 months ago (1 children)

Maybe just hard code the DNS value for archive.is in your host file or your pihole (if you use pihole)?

[–] _edge@discuss.tchncs.de 1 points 11 months ago (1 children)

Sounds like a simple solution.

Although I'm not really sure what happens here. I do get an IP address via quad9 and I do get other IP adresses using other resolvers, but how do I know which one works.

[–] redcalcium@lemmy.institute 1 points 11 months ago (1 children)

Both should work, archive.today is using a dns-based load balancer where it answer DNS query with an IP address for a server that supposedly closer to you. Just pick one with the shortest ping and see if it'll work.

[–] _edge@discuss.tchncs.de 1 points 11 months ago

But, then, why does it not work when using quad9? The result from quad9 may not be the closest server, but they can serve the captcha, so I'm reaching one of their servers.

[–] LWD@lemm.ee 1 points 11 months ago* (last edited 10 months ago)

deleted

[–] Spotlight7573@lemmy.world 1 points 11 months ago (1 children)

From my understanding, it's not quite the closest server:

https://news.ycombinator.com/item?id=36971650

I talked to the maintainer of archive.is years ago, they said this (hopefully they won't mind me posting):

There have been numerous attacks where people upload illegal content (childporn or isis propaganda) and immediately reported to the authorities near the IP of the archive. It resulted in ceased servers and downtimes. I just have no time to react. So I developed sort of CDN, with the only difference: DNS server returns not the closest IP to the request origin but the closest IP abroad, so any takedown procedure would require bureaucratic procedures so I am getting notified notified and have time to react.

But CloudFlare DNS disrupts the scheme together with all other DNS-based CDNs Cloudflare is competing with and puts the archive existence on risk. I offered them to proxy those CloudFlare DNS's users via their CDN but they rejected. Registering my own autonomous system just to fix the issue with CloudFlare DNS is too expensive for me.

So Cloudflare isn't doing anything wrong by passing DNS lookup results it gets from the archive.is servers to its customers instead of trying to 'fix' them somehow, but there does seem to be a somewhat legitimate reason for archive.is to be wanting the EDNS subnet information that Cloudflare does not provide due to customer privacy reasons.

[–] redcalcium@lemmy.institute 2 points 11 months ago

Returning not the closest IP, but the closest IP in a neighboring country? This is actually pretty smart. I wonder how effective it is at stalling takedowns though.