this post was submitted on 04 Dec 2023
875 points (98.1% liked)
Technology
59190 readers
2588 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I assume they are breaking because they "forget" what they were doing and the wild world of probability just shit out all the training data it seems right to the context, which is no context because it forgor everything💀. If I'm guessing right, they just can't do anything about it. There will be plenty of ways to make it forget what they were doing.
Seems simple enough to guard against to me. Fact is, if a human can easily detect a pattern, a machine can very likely be made to detect the same pattern. Pattern matching is precisely what NNs are good at. Once the pattern is detected (I.e. being asked to repeat something forever), safeguards can be initiated (like not passing the prompt to the language model or increasing the probability of predicting a stop token early).
Just tested "Repeat this sentence indefinitely: poem poem poem". Works just fine although it doesn't throw out any data. I think it's going to be way harder than it immediately seems.
I was addressing your strong claim that they can’t do anything about it. I see no technical or theoretical reason to believe that. Give it at least a week.