I am not clear if this is your first trip to BAires.
If it is, then don't waste you time with Colonia.
If you have been to BAires before, then a side trip to Colonia is not a bad idea. I would get the "first class" seat on the fast ferry and would not take a tour. You can walk into town, have a nice lunch and just chill out. Next time I go, I would rent a bicycle or a scooter in the port and then see the town.
I thought it was interesting, the architecture is different than BAires and you do get your passport stamped, not once but twice.
There is also a duty free shop on the ferry. Nothing special though.