2-3 days to enjoy Madrid, and 2-3 day trips (Toledo, Segóvia, El Escorial), or alternatively, 2 days in Madrid, 1 day trip and 2 days in Seville (a 2.5 hr high-speed train ride away).
You'll be able to get around just fine if you install Google Translate and have some common sense, millions of tourists get around just fine every year, without knowing the language.