You may be heavily underextruding on your 0.2mm setting, if it works at 0.1mm. Since the printer depends on extruding the right amount of material to build up to the nozzle tip, if you underextrude it won't build up high enough, and that 4th or 5th layer won't be close enough to the nozzle for the new plastic to be pressed down into it and stick.
Bump up those extrusion rates, slow down the movement feedrates, and make sure your nozzle and extruder motor are all clean and not slipping.