It's never good to get into it with other pax.

Easy solution has been to ring the call button, describe the situation to the FA, and that you'd like to avoid law enforcement intervention or a costly diversion.