That's right. The blue/grey ones are people who don't meet the criteria for influencing others' reputation scores. This is because they have fewer than 250 posts and/or have fewer than 20 reputation points (as per the FAQ). If they don't put a comment, then you have no idea whether it was a positive or a negative.

FewMiles..