this post was submitted on 04 Oct 2023
10 points (85.7% liked)
Free Open-Source Artificial Intelligence
2879 readers
4 users here now
Welcome to Free Open-Source Artificial Intelligence!
We are a community dedicated to forwarding the availability and access to:
Free Open Source Artificial Intelligence (F.O.S.A.I.)
More AI Communities
LLM Leaderboards
Developer Resources
GitHub Projects
FOSAI Time Capsule
- The Internet is Healing
- General Resources
- FOSAI Welcome Message
- FOSAI Crash Course
- FOSAI Nexus Resource Hub
- FOSAI LLM Guide
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Yes, but my understanding is that they are commutable? (i.e. the order does not matter) If so, it looks like that a "facts-adding" LORA seem to induce forgetting of formatting.
And I am especially curious if a facts-LORA + a instructions-LORA results in a model that can use the new facts in the instructions or not. I'll run experiments but would have loved if people here knew about it already.