Mechanically, a heat pump is an air conditioner with a reversing valve. In summer it moves heat out of the house; in winter it moves heat in.
Why it is efficient at heating
It does not create heat, it moves it. Even at low outdoor temperatures there is heat in outdoor air, and moving it costs less energy than generating it.
A heat pump delivering a COP of 3 supplies three units of heat per unit of electricity. Electric resistance heating is fixed at 1. This is why the comparison against baseboard or electric furnace heat is not close.
Where it clearly wins
Replacing electric resistance heat. The running cost difference is large and immediate, and this is the strongest case in the entire discussion.
Mild and moderate climates, where the outdoor temperature stays in the range where the heat pump is most efficient.
Where electricity is inexpensive relative to gas, which varies enormously by region.
Where there is no gas service, which removes the alternative entirely.
Where the case is closer
Against an efficient gas furnace in a cold climate with cheap gas. The heat pump may still be the lower-carbon choice and it may not be the lower-cost one. The arithmetic is local and it turns on the ratio of the electricity price to the gas price.
Where the electrical service is limited, since a heat pump with backup strip heat can require substantial capacity and sometimes a panel upgrade.
Cold weather performance
Capacity falls as outdoor temperature falls. Older heat pumps became marginal below about 35°F and leaned on electric strip heat, which is expensive and gave the technology its reputation.
Cold-climate heat pumps with variable-capacity inverter compressors hold useful capacity well below freezing, with many rated to deliver full heating capacity at 5°F and to operate below that. They are a materially different product from the equipment that reputation was formed on.
Dual fuel pairs a heat pump with a gas furnace and switches over below a chosen outdoor temperature — the heat pump handles mild weather at high efficiency, the furnace handles the coldest hours.
The sizing conflict
A heat pump must satisfy both a summer cooling load and a winter heating load, and in northern climates the heating load is much larger.
Sizing to the heating load leaves a system substantially oversized for cooling, with the humidity consequences that follow. Sizing to the cooling load leaves the heating short, made up by backup heat.
Variable-capacity equipment resolves most of this, because it can turn down far enough to cool correctly while carrying the heating capacity. This is a case where the equipment tier is not a luxury but part of making the arrangement work.
The practical questions on a replacement
Does the existing air handler suit it? A gas furnace paired with a heat pump becomes a dual-fuel system, which needs a compatible thermostat and control.
Is there electrical capacity for backup heat if it is needed?
Is the duct system adequate? Heat pumps deliver warm air at a lower temperature than a furnace — around 95–105°F rather than 130°F — so they move more air for the same heat. Ducts sized for a furnace can be undersized for a heat pump, and the symptom is a draught of air that feels cool even while it is heating.
Incentives
Federal tax credits and utility and state rebates for heat pumps are substantial and change year to year, and they frequently cover a large part of the incremental cost over a straight air conditioner.
Since the outdoor unit is being replaced regardless, that incremental cost is the only number the decision actually turns on — and with current incentives it is often small enough that the answer is straightforward.