An open jaw is fine with an amex voucher, the one restriction is that the jaw cannot be longer than any of the flights, so for example for LHR-XXX/YYY-LHR the distance for XXX to YYY must be shorter than LHR-XXX and YYY-LHR.

For your two examples trip 2 is fine.

Trip 1 (from gcmap)
LHR SYD - 10,573 mi
SAN LHR - 5,485 mi

SYD-SAN - 7,510 mi

So trip 1 doesn't work since SYD-SAN is longer than at least one of the flights.