Cheap = text editor.

Depends on what you want to do. Over at my personal website ben-johnson.org, I'm running a weblog powered by WordPress (wordpress.org), and a photo gallery run by (the aptly named) Gallery (gallery.sourceforge.net). Check them out to see what you can do.

To do these software packages, you'll need a web hosting account, which can typically be had for <$10/month with a decent chunk of storage space and bandwidth. I'm using a company called DreamHost (dreamhost.com), and they'll throw in a free 1-year registration of your domain name (usually an $8-10 value).

If you're just looking for software to write HTML, obviously, a text editor will be the cheapest way to go. You can do HTML in MS Word, although it's not very elegant. Then, there's dedicated software like DreamWeaver, but I don't know pricing for that off the top of my head.