1- He is under 18.. does that make a different ?
No.
2- The total cost of the items is still within limit, why he had to pay fees .. ?
Because he lied would be my guess.
3- I am more concerned about him being put on a list where they will give him hard time every time he passes the boarder, is it the case .. ?
If the guy who rents a room from you is giving your brother a hard time, I suggest eviction.
4- If we need to contest this, what is the proper channel, how do we go with this .. ?
Try
here. In what way did the officer apply the law incorrectly or misunderstand the facts?