23
submitted 1 year ago* (last edited 1 year ago) by RankWeis@sh.itjust.works to c/piracy@lemmy.dbzer0.com

Is there a way to speed up extraction of 40+gb 4k videos? This is by far the most time consuming part of the process and I was wondering if there was any hardware acceleration or even multi cpu way of speeding this up.

My cpu isn’t the newest, it’s an i7 5820k, overclocked just a bit, but I have a hard time believing that a newer cpu on its own will have more than marginal gains on this front, but I could be way off here.

edit: I just realized my processor came out in 2014. So I'm sure in 10 years we've made some inroads, I thought I bought this guy in 2018.

top 16 comments
sorted by: hot top controversial new old
[-] Faceman2K23@discuss.tchncs.de 10 points 1 year ago

The unpack process is usually single threaded, and a stock 5820k is slower than a ryzen 1600 in that case, so you're probably just running out of CPU performance. My server runs a 6700k and it's pretty slow to unpack even with nvme arrays

[-] OsrsNeedsF2P@lemmy.ml 9 points 1 year ago* (last edited 1 year ago)

Excluding everything that is more difficult to deal with (like getting an SSD/more RAM/more swap on Linux), have you tried using different extraction apps? Peazip is good: https://peazip.github.io/

Edit: Here are some benchmarks for fastest extraction, Peazip is very good https://peazip.github.io/peazip-compression-benchmark.html#7-zip_vs_winrar

[-] RankWeis@sh.itjust.works 2 points 1 year ago

I have not, I will give it a go, thanks!

[-] PeachMan@lemmy.one 6 points 1 year ago* (last edited 1 year ago)

Your drive speed might be the limiting factor here, not your processor.

[-] RankWeis@sh.itjust.works 2 points 1 year ago

Interesting, I'm using a synology NAS in order to store lots of data, it should be relatively speedy but probably not compared to an onboard SSD. My SSD only has about ~100gb of free space so even using it as a temporary download center may not be right, looks like I'm looking into a terabyte nvme. Thanks for the suggestion!

[-] crossover@lemmy.world 4 points 1 year ago* (last edited 1 year ago)

Are you sure network isn’t a bottleneck? Even with a gigabit network, copying 40gb worth of data back and forth can take a while.

[-] randombullet@feddit.de 4 points 1 year ago* (last edited 1 year ago)

Yeah network speed for gigabit is max 125MB/s. Or closer the 115MB/s including network overhead.

Even with 2.5G networking I'm only hitting 250MB/s for my NAS. I've never seen anything close to 300MB/s

[-] PeachMan@lemmy.one 2 points 1 year ago

To be clear, I'm not saying that I'm sure your bottleneck is the SSD. But it could be either that or the processor.

[-] entropicdrift@lemmy.sdf.org 4 points 1 year ago

If you're using the *arr apps to download, consider using unpackerr because it'll handle extraction and cleanup in the background.

[-] RankWeis@sh.itjust.works 2 points 1 year ago

I am, this is cool, thank you!

[-] nicman24@kbin.social 4 points 1 year ago

tf are you getting 4k torrents that are ziped and why??

[-] RankWeis@sh.itjust.works 3 points 1 year ago

It’s usenet, everything is zipped into hundreds of RAR files

[-] sin_free_for_00_days@sopuli.xyz 3 points 1 year ago

What type of speed is your hard drive?

[-] RankWeis@sh.itjust.works 2 points 1 year ago

It's hard to say since it's all RAIDed out but they're not SSD's, I just didn't have the space on the drive. But looks like I'm in the market for a new cpu, motherboard and NVMe drive now. Thanks for your input!

[-] grasib@lemmy.world 4 points 1 year ago

Are you unpacking them directly on your NAS?

Because chances are, that if you use a Synology and map the drive or use the GUI, that the data keeps getting transferred between your PC and the NAS. This is a very slow process, especially if you connect by WiFi.

[-] Pulp@lemmy.dbzer0.com 3 points 1 year ago

Some players can play rar files directly. Also look into rar2fs. I just try to avoid rars if possible

this post was submitted on 13 Aug 2023
23 points (87.1% liked)

Piracy: ꜱᴀɪʟ ᴛʜᴇ ʜɪɢʜ ꜱᴇᴀꜱ

54443 readers
268 users here now

⚓ Dedicated to the discussion of digital piracy, including ethical problems and legal advancements.

Rules • Full Version

1. Posts must be related to the discussion of digital piracy

2. Don't request invites, trade, sell, or self-promote

3. Don't request or link to specific pirated titles, including DMs

4. Don't submit low-quality posts, be entitled, or harass others



Loot, Pillage, & Plunder

📜 c/Piracy Wiki (Community Edition):


💰 Please help cover server costs.

Ko-Fi Liberapay
Ko-fi Liberapay

founded 1 year ago
MODERATORS