If you ever plan to get a job requiring a US security clearance, any "below board" travel to Cuba will probably nix being able to get the job. That's unreasonable/ unfair by my personal yardstick, but that's the reality of current laws which I hope will change.
Rest assured, all your travel is known about. A friend recently had his security clearance upgraded to work on a new government contract that his employer had been awarded. During a routine but lengthy interview to process his clearance, they focused on domestic flights he had taken in SE Asia on a vacation, as well as local phone calls he had made to arrange and confirm local tours and flights. This isn't a rant or paranoia with Snowden at the top of the news...it's been reality for decades.
So, I'd recommend being totally above-board and complying with US laws if you or any family members may need to get a security clearance. Agree with others that the Treasury or State Departments are probably not a concern. Again I don't agree with our Cuba policy at all.