There is now that super direct train from Penn Station. Take the Subway from Grand Central to Penn. Use the S to Times Square, then follow the signs to the 1 and 9 downtown. It's like one or two stops on either the 1 or 9.