9
submitted 4 months ago* (last edited 4 months ago) by ylai@lemmy.ml to c/stablediffusion@lemmy.ml

Excerpt from the relevant “ComfyUI dev” Matrix room:

matt3o
and what is it then?

comfyanonymous
"safety training"

matt3o
why does it trigger on certain keywords and it's like it's scrambling the image?

comfyanonymous
the 2B wasn't the one I had been working on so I don't really know the specifics

matt3o
I was even able to trick it by sending certain negatives

comfyanonymous
I was working on a T5 only 4B model which would ironically had been safer without breaking everything
because T5 doesn't know any image data so it was only able to generate images in the distribution of the filtered training data

comfyanonymous
but they canned my 4B and I wasn't really following the 2B that closely

[…]

comfyanonymous
yeah they did something with the weights
the model arch of the 2B was never changed at all

BVH
weights directly?
oh boy, abliteration, the worst kind

comfyanonymous
also they apparently messed up the pretraining on the 2B so it was never supposed to actually be released

[…]

comfyanonymous
yeah the 2B apparently was a bit of a failed experiment by the researchers that left
but there was a strong push by the top of the company to release to 2B instead of the 4B and 8B

Additional excerpt (after the Reddit post) from Stable Diffusion Discord “#sd3”:

comfy
Yes I resigned over 2 weeks ago and Friday was my last day at stability

no comments (yet)
sorted by: hot top controversial new old
there doesn't seem to be anything here
this post was submitted on 17 Jun 2024
9 points (100.0% liked)

Stable Diffusion

1487 readers
1 users here now

Welcome to the Stable Diffusion community, dedicated to the exploration and discussion of the open source deep learning model known as Stable Diffusion.

Introduced in 2022, Stable Diffusion uses a latent diffusion model to generate detailed images based on text descriptions and can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by text prompts. The model was developed by the startup Stability AI, in collaboration with a number of academic researchers and non-profit organizations, marking a significant shift from previous proprietary models that were accessible only via cloud services.

founded 1 year ago
MODERATORS