"Computer Says No" as an excuse to deny people social services on an ostensibly meritocratic basis is a real problem, but it is one that already exists.

And while statistical bias is easy to echo with AI, its black box nature is traitorous. For instance, the UK home office inadvertently began flagging passports from Albania, Greece, Romania and Bulgaria as participants in "sham marriages". This created enormous volumes of extra paperwork for no discernible benefit.

Rather than being racist, you mostly just establish yourself as unreliable.

The enshittification will be bad, but it will also inhibit large scale adoption.

Case in point, the guy who tricked Chevy into selling him a Tahoe for $1.

Or the Dallas Judge who prohibited AI generated brief in the Fifth Circuit on the grounds that they hallucinated cases.

If only jobs like this could reliably be automated. But as someone who works in automation, I can tell you a whole litany of reasons why they aren't going to achieve that any time soon.

If nothing else, these industries simply aren't set up to interface with AI in a functional capacity. So much of a creative career is predicted on who you know that guys like Zach Snyder will never be seriously threatened by an AI director. So much of a physician's job is social rather than analytical that no amount of WebMD wish casting will replace them.

Ooof I hadn't even thought about that. I mean with a lot of human interactions playing out on screen, it's already setting expectations of what people should be like. Using AI generation to build on an exclusionary body of work makes that even worse.