Originally Posted by
hdogan
In theory, you should get your Schengen visa from the country that will be your "main destination", or where you will spend the majority of the time. In practice, most likely nobody will care, once you enter Schengen, you can freely move around. If you want to be safe about it, make your Poland stay longer than your Latvia stay

As stated above if you can get easily the Polish visa better you enter Poland and then you can move to any other Schengen country, you can leave Schengen area from Latvia, no need to go back to Poland unless required for other reasons.