BA Holidays terms and conditions include the following clause:
8.3. We are only liable to you for cancellations or significant changes to your booking by us. A significant change is one that we make to your booking before your departure that affects an essential term of your contract. Examples of a significant change includes changes by us to:
- your destination area;
- your booked accommodation to that of an alternative property of a lower rating;
- your departure times or accommodation occupancy periods by more than 12 hours.
So, basically, they can change your departure time by up to 12 hours either way without having to refund you.