A heat pump will always generate a small amount of heat just from the compressor running, but most of the time that's a lot less energy than is being moved. As the outdoor temperature drops the delta between input and output air temp will decrease until the difference is entirely from generated heat in the compressor. Most designs would turn on extra resistive heating once the output temperature drops below your set target though. Modern designs are capable of moving a reasonable amount of heat even down to at least -25°C / -13°F now though.

It gets less effective, down to running at 100% and not moving heat. Heat pumps work by expanding a gas, which cools it. Since it's cold, the "heat" outside was the gas. Then the gas is taken inside and compressed, the gas heats up from the compression (since all the energy is squeezed into a smaller space, effectively speaking). Now that heat can be transferred to the colder air inside. So long as the expanded gas turns colder than the outside, it can absorb heat.

From a Google, common ones can go as low as - 25C, which means they are able to cool a gas to lower temps than that when expanded. There is still heat to get, even in -25C.

One thing that happens is that the defrost cycle takes a longer time, so it spends less time heating the building

I can only talk for myself but I have a Nibe pump here in Sweden with air source outside pump and water heating system to radiators on the inside. Even down to -30° with really shitty windows it was enough heat for me to be comfortable. Though it did indeed use the supplementary resistive heating a bit it was still able to give me about a 200% efficiency during that period. Give a typical winter (usually around -5-10C but, as said can go down to -20C or -30C for a week or so) it still runs an average of about 300-400% efficiency.

I have an air sourced heat pump and it gets to -35C for a few weeks at time here. When it's that cold it does produce heat but your breath is hotter. There's no point in running it as it just doesn't make any kind of useful heat. Below -10C the amount of heat it produces noticeably tapers off.

You either buy some portable electric heaters for those 2-3 weeks when it's necessary, or you get a heat pump that has resistive heating as a backup.