When watching the TV, I found the controls on the arm rest quite annoying. On more than one occasion I ended up changing the channel with my elbow and I could imagine if you are in the middle seat, it being worse.
You are absolutely right. This is a major design flaw with the controls where they are on F9 birds.