Before I got to the part where you already stayed at Viceroy Anguilla, I was thinking that fits your criteria with the modern/contemporary. We enjoyed Jumby Bay and thought seven nights was almost a little too long (started to feel a bit stir-crazy) so a long weekend would be a nice match if the pricing shifts. However, nothing about the suite was even remotely the feel of a modern or contemporary, it felt traditional. Never spent any time at the pool, so can't speak to that experience. I'm not sure if you will be happy there though if modern/contemporary is of strong importance.
If you don't mind going back to Anguilla, have you looked at Malliouhana? We haven't stayed there overnight yet, but the pool looked beautiful and has an ocean view. Former Aman is the GM last I checked. The restaurant is what kept us from booking last year, but it should be much better with Marc Alvarez taking over. Not as modern in architecture as Viceroy/Four Seasons, but certainly more contemporary than most.