Can't really fault the FA that much as they aren't in the position to know what really happened and have to take both sides' stories at face value.

On a purely rational level it makes sense for the airline to move the guy in that scenario. If that lady writes a complaint going: "A pervert stared at my breasts as I was breast-feeding on my flight and Emirates ignored it even though I complained!", true or false, the possible impact is gonna be a lot higher than some dude going "I got relocated against my wishes".