I detect a bit of BA's bias coming out here.
I wouldn't dream of advising someone who may never visit this part of the world again not to fit in a trip to Colonia even though they only have three days in BA.To be honest they could have three weeks in BA and not have enough time to take it all in so using one of their three days to sample another country is fair game in my book and won't make much difference to their memories of the city.
Personally, I would rather fly over to Montevideo for the day and get a truer sample of Uruguay life there.