I never worry about turbulence during flight. Just as you are bound to hit a bad stretch of road on a long road trip, you are going to run into a couple bad turbulence on a trans-continental flight.
The only time I worry about turbulence is during landing. With not a whole lot of room underneath the plane, if a sudden microburst pushes the plane downwards, there is not much a pilot can do to recover. That is why I get nervous everytime the plane comes in for landing in a driving storm.