I don't think there is a minimum size. I've always thought it would be funny to check a penny. Since the bag tags have adhesive on the back (remember when they used to be real tags?), you could stick the penny on the back and it would be essentially just the bag tag.

Unless your toiletries are expensive, I doubt the effort to get a box isn't worth the value of the toiletries should the plastic bag get ripped. We've bent over far enough as it is; let's not try to make things too easy!

One time I checked my backpack in a plastic bag because I wanted to keep my pocketknife (tried to take it through security but they found it on the X-ray machine; 1 for 2, darn!) The bag survived intact.