I think my cognitive dissonance was too strong, I got rid of my Gaiman. :-(
But I feel you - his works were important in my life before, I've just been downsizing and even though it wasn't the best, I decided to get rid of mine (not because it's "right" but just because I don't like being reminded of him).
tbh my feelings seem to be guiding things before anything like rational morality does - I feel cognitive dissonance about his art because of the association with him as a rapist, and that's enough for me to ditch his art without having to justify it as a moral necessity that others must do as well.