I've tested it and the original user does have the option to delete the PM, and it does disappear from your mailbox even if the user is blocked. But you say they were already banned. If it was a lemmy.world user, then there is probably some way that lemmy.world admins can manually take over that account and delete the message from the fediverse, but the message you are seeing is the local sh.itjust.works copy, so sh.itjust.works admins should have the power to delete it even if lemmy.world falls into a black hole.