it was never a slogan for me

For the banks financing the house?

I wouldn't call them middlemen, I would say the banks are just a third-party to the renter-rentee relationship which is direct exploitation. The banks are also exploiting, but in different ways, they're exploiting their low level processor employees and bank tellers and the like in their own employer-employee relationship. If they are charging usurious rates on the mortgage, they are exploiting the lender-lendee relationship with the landlord. These are all separate examples of exploitation, but we believe all the exploitations should be stopped not just 2 of 3.

Also middlemen in exploitative relationships aren't a good thing. "Cut out the middlemen" is still good practice under communism if they aren't contributing anything. A hitman is just a "middleman" in the relationship between the victim and the one buying the hitman service. Doesn't mean hitmen shouldn't be abolished just like purchasing hitmen.