Reply From Uncle Agony
If you are in Singapore for less than 181 days per year, you are not within the tax jurisdiction of the jurisdiction and hence all your income wherever it is denied are not taxable in Singapore.
However it seems that from your contract it is your duty to pay your Sri Lanka tax yourself. If your company had indeed paid the Sri Lanka Department of inland revenue on your behalf it is only right that your repay them but make sure you show proof before you part with your money.
Two years in-ordinate delay is not a time bar to them claiming from you.
However the terms of your agreement might have overridden any obligation on the part of your company to fulfill their responsibility to pay your PAYE tax at the time they pay your remunerator. So you might have to repay them.
To advise you fully one would require your employer’s handbook and the complete employment agreement.