1472
Dreams of AI
(lemmy.world)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 03 Jun 2024
1472 points (97.9% liked)
People Twitter
5171 readers
1005 users here now
People tweeting stuff. We allow tweets from anyone.
RULES:
- Mark NSFW content.
- No doxxing people.
- Must be a tweet or similar
- No bullying or international politcs
- Be excellent to each other.
founded 1 year ago
MODERATORS
Do you understand how hard that is? Most websites do not just let you scrape them entirely because they think you're a competitor. Web scraping involves staying one step ahead of each site. You might as well just pick a site and buy most things there.
Plus you can just have everything delivered. It's easier and you can get access to sales.
Yes, I do indeed, for example the AutoTL;DR bot you might have seen around Lemmy (that does web scraping) is mine.
Sure, you can do an entirely different thing than I was talking about, but then you're replying to something entirely else, aren't you? This was a reaction to a post in the OP which doesn't talk about getting everything delivered.
There are 3rd party APIs that handle scraping.... Eg Red circle API. Problem is they don't really have produce and everything up to date.