There's a bit more to it, but it's because of this effect.

There is actually a balance between liquid and gas state, just overwhelmingly in favor of liquid when at normal temperatures. There is a ratio of molecules that will hit each other and transition to gas, and an equal amount gas hitting liquid and condensing. At least when there is a balance between the two sides, aka 100% moisture in the air. Which is not how it is most places.

Normally there is always evaporated water in the air, and anything that evaporated will be moved away in any mildy ventilated area, as you say, it leaves the system. So it never reaches a balance, which is why things dry up at lower temps as water will always evaporate and leave the system.