158
Bill Gates feels Generative AI has plateaued, says GPT-5 will not be any better
(indianexpress.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 02 Dec 2023
158 points (85.6% liked)
Technology
59118 readers
3148 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
ChatGPT doesn't learn like that though, does it? I thought it was "static" with its training data.
You can finetune LLMs using smaller datasets, or with RLHF (reinforcement learning from human feedback) wherein people can give ratings to responses and the model can be either "rewarded" or "penalized" based off of the ratings for a given output. This retrains the LLM to produce outputs that people prefer.
Active Learning Models. Though public exposure can eaily fuck it up, without adult supervision. With proper supervision though, there's promise.
So it will always have the biases of the supervisors
Bias is inevitable. Whether it is AI or any other knowledge based system. We just have to be cognizant of it and try to remedy it.
I was speculating about how you can overcome hallucinations, etc., by supplying additional training data. Not specific to ChatGPT or even LLMs...