this post was submitted on 05 Jan 2024
280 points (97.6% liked)

Selfhosted

40383 readers
569 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz
280
13 Feet Ladder (github.com)
submitted 10 months ago by tree@lemmy.zip to c/selfhosted@lemmy.world
24 comments fedilink hide all child comments
 

A site similar to 12ft.io but is self hosted and works with websites that 12ft.io doesn't work with.

How does it work?

It pretends to be GoogleBot (Google's web crawler) and gets the same content that google will get. Google gets the whole page so that the content of the article can be indexed properly and this takes advantage of that.

link: https://github.com/wasi-master/13ft

top 24 comments
sorted by: hot top controversial new old
[–] KLISHDFSDF@lemmy.ml 85 points 10 months ago (3 children)

If you're on Firefox on desktop/laptop, check out Bypass Paywall [0]. It was removed from the firefox add-on store due to a DMCA claim [1], but can be manually installed (and auto updates) from gitlab. The dev even provides instructions on how to add custom filters to uBlock Origin [2], so you don't have to add another extension but still get some benefit.

[0] https://gitlab.com/magnolia1234/bypass-paywalls-firefox-clean

[1] https://winaero.com/mozilla-has-silently-removed-the-bypass-paywalls-clean-add-on-from-amo/

[2] https://gitlab.com/magnolia1234/bypass-paywalls-clean-filters

[–] hdnsmbt@lemmy.world 22 points 10 months ago (1 children)

Your correct indexing is highly appreciated!

[–] desmosthenes@lemmy.world 6 points 10 months ago (1 children)

took the words right out my mouth

[–] AtariDump@lemmy.world 4 points 10 months ago

It must have been while he was kissing you.

[–] ssdfsdf3488sd@lemmy.world 3 points 10 months ago

That's the dude who was butt hurt about something this dude did: https://github.com/iamadamdev/bypass-paywalls-chrome

and so forked it and arguably does a better job, lol.

[–] ASeriesOfPoorChoices@lemmy.world 2 points 10 months ago

also, bypass paywalls clean on notfirefox, like Chrome, or Kiwi (android).

[–] Qwaffle_waffle@sh.itjust.works 39 points 10 months ago (2 children)

Where are the metric versions? I want my 3 meter ladder.

[–] MacNCheezus@lemmy.today 5 points 10 months ago

Clone the repo and make it yourself.

[–] spechter@lemmy.ml 2 points 10 months ago

Most often I use it, it's too avoid metrics.

[–] redcalcium@lemmy.institute 28 points 10 months ago* (last edited 10 months ago) (2 children)

It amazes me that all it takes is just changing user agent to Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) and it can bypass paywalls on many sites? I thought those sites would try harder (e.g. checking if the ip address is truly belong to google), but apparently not.

[–] andrew@radiation.party 12 points 10 months ago (1 children)

Checking ip ownership is a moving target more likely to result in outcomes these sites don’t want (accidentally blocking google bots and preventing results from appearing on google).

Checking useragent is cheap, easier, unlikely to break (for this purpose, anyway) and the percentage of folks who know how to bypass this check is relatively slim, with a pretty small financial impact.

[–] douglasg14b@lemmy.world 2 points 10 months ago (1 children)

It's not necessarily a moving target when entire blocks can be associated with Google.

[–] andrew@radiation.party 3 points 10 months ago (1 children)

Unless they are permanently only using specific addresses or blocks and will never change that up, I’d consider it a moving target.

[–] efstajas@lemmy.world 1 points 10 months ago

Google literally has an official list of IP ranges for their crawlers, complete with an API that returns the current IP ranges that you can use to automate a check. Hardly a moving target, and even if it is, it doesn't matter if you know exactly where the target is at all times.

[–] aniki@lemm.ee 2 points 10 months ago

Same. I thought there would be more stuff happening in the background but when I saw it's just hijacking the google bot headers to display the html i was a bit disappointed it's so stupidly easy.

[–] thanksforallthefish@literature.cafe 17 points 10 months ago (1 children)

Https://1ft.io also seems to work and by the branding seems unrelated to 12ft

[–] cyclohexane@lemmy.ml 3 points 10 months ago* (last edited 10 months ago)

There's 4ft.io too. Oh nvm looks like it's gone.

[–] appel@lemmy.ml 16 points 10 months ago

If you're on Android and use Firefox, you can use the Disable JavaScript extension to disable JS on sites with paywalls, like NYtimes. While not perfect, it works remarkably well.

Also works great on Desktop.

[–] SeeJayEmm@lemmy.procrastinati.org 9 points 10 months ago* (last edited 10 months ago) (1 children)

I've been happy with https://github.com/everywall/ladder

[–] tootbrute@piefed.social 2 points 10 months ago

I use this, too! It's great but doesn't always work.

[–] CCMan1701A@startrek.website 4 points 10 months ago

Loaded the docker for fun on my NAS. I don't need it, but other users in my home may appreciate this.

[–] jadelord@discuss.tchncs.de 3 points 10 months ago

Seems like this can be done in the browser using a user agent switcher.

[–] knova@infosec.pub 2 points 10 months ago

Love it! Deployed it this morning.

[–] PlexSheep@feddit.de 1 points 10 months ago

So ist this an http proxy? I don't quite get it.