That's not actually what OP meant by this, but I understand why you might get in this line of thinking. I used to embody it myself.

Look at it this way: the categorization and the way we handle sex is definitely socially constructed. The very fact that we made sex a "thing" using these categories is the result of social construction. Abolishing sex or gender isn't making the case that everyone must be androgynous or gender-neutral in presentation or in any aspect, but rather, it's asserting that we shouldn't assign any specifics relating to presentation, role, or lifestyle to these rigid notions we've developed in the context of sex and gender.

What we now know as "gender dysphoria" could still exist if gender were to be abolished. What we now know as "masculinity" and "femininity" could still exist if gender were to be abolished. What we now know as "biological sex" could still exist if gender (or sex, depending on how you want to frame the semantics) were to be abolished. It's just that the societal outlook and approach to these things would change entirely, in the sense that these things would be about as gendered as attached earlobes and free earlobes are. Believing that these things will remain isn't mutually exclusive with the abolitionist view that OP offered, but like I said, I understand why you hold this viewpoint because I held it myself for a very long time.

It only became clear to me what OP was conveying when I read The Gender Accelerationist Manifesto, and with that, I unlocked the realization that my experience, especially as a non-binary trans person in particular, would benefit from nothing more than to not have gender matter at all in my life, and with that, the grand scheme of society as a whole.