Three hours would be tight. Remember that you need to factor in waiting for your bags to be unloaded at the airport. You mentioned "big bags." I doubt carrying them on will be an option on your flight to CUZ.
Since you'll be pressed for time and since your Spanish is limited, I'd have your hotel arrange the car. That way, there is someone waiting there to whisk you off to Ollantaytambo.