Remember: One ticket on one airline will always be cheaper than two airlines on one ticket.

So if you want a savings and same plane service, go with QR only and forget about the change of plane in BKK connecting onto Bangkok Airways. If you go entirely on QR, the plane just stops in KUL to refuel and pick up passengers. You do need to get off the plane. So stretch your legs a little but you do not need to go through immigration and customs since you're not stopping in Malaysia.

If you go the way via BKK, you might have to get off from BKK, claim your bags , go through immigration and then recheck onto a domestic flight. Plus this option, although it might save you a couple of hours, is more problematic.