After getting a residence permit in Latvia, do I have to live in the country in order to get it renewed at the end of the validity period?

There is no physical presence requirement if you just keep the Temporary Residence Permit (TRP). Also no physical presence requirement to renew the TRP after five years for another five years. TRP is granted for five years, but the eID card is renewed on an annual basis. The card renewal takes three days and is in person, so for that you will be required to go to Latvia. However, if you want a PR, after five years of having the TRP, then you will need to reside in Latvia.