Interest in a Kemono/Coomer scraper? (lemmy.world)

submitted 1 year ago* (last edited 1 year ago) by Set8@lemmy.world to c/piracy@lemmy.dbzer0.com

5 comments fedilink hide all child comments

Hello Lemmy piracy subreddit,

Posting this here as the reddit sub has a karma threshold which I don't meet and can't be bothered to pander to.

I've written 2 GUI-bases programs that were originally CLI as a personal project. These programs allow you to batch download posts from a single creator on the party networks Kemono and Coomer. Both feature an automatic translation toggle allowing all posts and the creators name to be translated (uses Google translate API) and has various config options.

Considering this was a relatively simple and straightforward tool, I feel it's a bit niche and thus am asking the community whether they would like this program to be released. If enough people want it, then I'll release it publically as long as the kemono and Coomer devs are fine with it. The tools don't really do any aggressive behaviors as it mimics an actual user as much as possible (selenium is not used here for the nerds reading, only raw GET requests) and doesn't tax kemono/Coomer servers to my knowledge. If this is incorrect then I obviously will revoke the release if applicable.

Please let me know in the comments if this is something you'd be interested in using, or if this is too niche of a program.

top 5 comments

sorted by: hot top controversial new old

[-] rikonium@discuss.tchncs.de 4 points 1 year ago* (last edited 1 year ago)

I think the cats out of the bag already - I’ve seen a “coom dl” GUI and something CLI in the wild but not sure if they handle Kemono

[-] Set8@lemmy.world 5 points 1 year ago* (last edited 1 year ago)

From a programming standpoint kemono is a bit harder to pull off because they use a lot of archivers and have more foreign creators as the most popular results

However yes, it's a relatively simple program from a programming standpoint, however I tried to make the GUIs as QoL as possible with multithreading and a file downloading lib that allows fast and efficient multipart downloading which circumvents a lot of RAM requirements if the entire download is stored into RAM (such is the case with the C# REST library)

Another benefit of kemono is that it isn't STRICTLY for pornographic content and has a variety of patreon files which my scraper also supports. Currently only discord is unsupported at the moment solely because parsing the HTML is extremely annoying as they basically piggybacked the actual discord service's URL scheme