When I did the trip (a couple of years ago) I just booked it at the railway station (in Bergen in my case). That is pretty much the best way to do this. We did a round trip Bergen (by boat) Flam (by train) Bergen.