You can, at least in theory, use the BA lounges. However, there are issues with this:

1. In most locations, the AA and BA gates are not in the same concourse/terminal. Will TSA let you through to the BA lounge with an AA boarding pass? If not, then no access at places like JFK or LAX (where the AA lounges are pretty decent anyway), at ORD (where the BA lounge over in T5 really isn't that good), EWR, BOS and SFO.

2. Unlike the US airlines, BA isn't usually set up to have a list available at security of people who are allowed to pass to access the lounge - there are no conference rooms in the BA lounges and unlike Admirals Club membership lounge access is not something you have paid for. Nonetheless, I can think of places like DEN and DFW where the BA lounge is in an area you would be able to access.

2. Opening hours. BA lounges often only open for the BA departures.

3. Capacity. From experience I can tell you that some lounges (DEN, SEA) are full around BA departure times. They are able to refuse access based on capacity grounds, and would not be surprised to have this rolled out as a reason to deny you entry in favour of the Admirals Club.

In short, if you are in the right airport at the right time then I don't see why you can't try the BA lounges. Do let us know how you get on!