You can use points to book flights within Europe, but can probably only have one stop-over.
You might try something along the following lines:
Book a transatlantic award YVR-CPH with stop-over in LHR then return from MAD-YVR. (I am assuming that you can do a stop-over AND an open jaw - could be wrong in which case you'd need to just do the open jaw.
Then use points for flight within Europeby booking CPH-HEL and HEL-ZRH with a revenue ticket from ZRH-MAD or a revenue ticket from CPH-HEL then book HEL-ZRH and ZRH-MAD
If they don't allow both open jaw and stop-over, book YVR-LHR then LHR - CPH with a return CPH - YVR then use a Europe reward to go from HEL-ZRH, ZRH-MAD and MAD-CPH.
As an alternative, have you thought about buying a revenue ticket and then buying a Star Alliance Europe Airpass for your flights within Europe?