This is no longer the case. The visa in advance requires a letter of invitation and the cheapest they come is $100. You then need to pay for the visa and postage, etc. on top of that.

Visa on arrival is still available at Heydər Əliyev Airport although the Azeri Government has said it will remove the facility. When that happens, it is likely to be with no notice, and since it takes at least 3 weeks to get the visa in advance, that could leave some people in an awkward spot.

Armenian visas are not a problem, but a Nagorno Karabakh visa would be.

Baku is an expensive city and there are plenty of traps for unwary visitors - especially tea in a seafront cafe which will lead to an eye watering bill. A few words of Russian will help a lot.