They'll let you know right away. It doesn't happen often but it does occur due to weight limitations. The alternative is to fly to Boston.
I fly YYZ-PVD regularly throughout the year and have never seen this happen. That said, it IS a tiny plane so I wouldn't rule it out should the Beechcraft be at full passenger capacity.
The flight to BOS plus the drive into Rhode Island isn't too bad. You can always stop for lunch/dinner in Boston before continuing into Providence.
