I'm sitting here looking at my vintage Olympus OM-1. Love shooting on that thing. Would keep doing it if I could find somewhere locally to develop the film.
The bodies are relatively cheap. You can find good condition stuff online for maybe $100-200 tops. The film however, not cheap, especially if you're into specialty film (B&W, Infrared, etc. usually runs $10-14 a roll or significantly more if you're looking for vintage film).
There's also a fairly healthy secondhand market for vintage glass, so picking up a lens secondhand shouldn't be too much of an issue.