Interest payments made on government debt hit the highest level on record for May as inflation surged.