Society has to pay any healthcare stuff that’s caused by smoking. An earlier death and the resulting savings for the retirement fund is not enough to outweigh that.

Actually, it likely is. What you're forgetting is that smoking also saves a lot of money in healthcare. After all non-smokers also age and get sick. The early death of smokers due to smoking related diseases prevents a lot of costs due to age related diseases. Afaik there's no consensus about whether or not smoking saves money on healthcare. E.g. in this study smoking costs money with males, but saves money with females. If you count in pension then the additional healthcare costs are easily offset.

I am however fairly sure that smoking becomes expensive for society when you start to include productivity. Smokers tend to contribute less to society, though afaik it's hard to calculate the effect of that because smoking is associated with lower education.