What if the next available flight after your missed connection is long enough in the future that the resulting delay means that you miss something that you had scheduled to do at your destination?

Actually, this is probably the most relevant criterion in determining how bothersome a flight delay is. If a flight delay does not cause you to miss something you intended to do upon arrival, it is much less of a problem than if the flight delay does cause you to miss something (business meeting, wedding, funeral, cruise ship departure, etc.).

An additional criterion would be if the delay imposed some unreimbursed cost to you, such as requiring an overnight hotel stay due to a missed connection on the way home.