Archiveteam Veoh grab (tracker.archiveteam.org)

submitted 3 days ago* (last edited 1 day ago) by kabi@lemm.ee to c/datahoarder@lemmy.ml

7 comments fedilink hide all child comments

Just looking at the numbers, it doesn't seem to me like archival will complete before the shutdown date (nov. 11). There are 2million+ elements left, likely 100TB+ of videos.

If you care to help them out, see instructions at the top of the page. Be sure you have a "clean connection", though.

edit: They're saying that the current rate seems to be plenty enough to finish by the deadline. Workers are often left idling at the moment.

top 7 comments

sorted by: hot top controversial new old

[-] clb92@feddit.dk 3 points 1 day ago* (last edited 1 day ago)

I've started a few warriors, but it's not helping because they've activated rate limiting. I just get:

Tracker rate limiting is active. We don't want to overload the site we're archiving, so we've limited the number of downloads per minute. Retrying after 240 seconds...

So the bottleneck is that they don't want to overload Veoh.

[-] kabi@lemm.ee 1 points 1 day ago

They are about to double the rate lmit, so it should be a little better...

[-] kabi@lemm.ee 1 points 1 day ago

Yeah, that's what I'm getting now, too. Hopefully they take the deadline into account for rate limiting... Still good to have warriors ready to retrieve, I guess.

[-] ReversalHatchery@beehaw.org 3 points 2 days ago* (last edited 2 days ago)

thanks for the reminder! recently I keep the warrior down because my amount of ram started to be a bottleneck to me, but certainly manageable when there's urgent need.

why don't they switch the "current project" selection to it, though? It's on telegram now. it would receive more help because that's the automatic choice

[-] Kissaki@beehaw.org 1 points 2 days ago

I tried getting the warrior to work with Hyper-V. But that doesn't seem possible or feasible.

[-] Kissaki@beehaw.org 1 points 2 days ago

Why does warriorhq link to warrior version 3 when there's a successor version 4 available?

[-] kabi@lemm.ee 1 points 2 days ago

Not sure. They don't mention 4 being in beta or anything. Though they don't list much advantage of using it over 3 either.

this post was submitted on 01 Nov 2024

9 points (100.0% liked)

datahoarder

6699 readers

3 users here now

Who are we?

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

We are one. We are legion. And we're trying really hard not to forget.

-- 5-4-3-2-1-bang from this thread

founded 4 years ago

MODERATORS

archivist@lemmy.ml