Tay joined the Singapore Armed Forces (SAF) on 21 December 1981, and was awarded the SAF Overseas Scholarship in 1982. He holds a Master of Arts degree in engineering science from the University of Oxford, and a Master of Science in management from the MIT Sloan School of Management. Prior to joining the RSN, Tay studied at Anglo-Chinese School and Anglo-Chinese Junior College. Tay is married to Leah Teo and they have two sons, Reuben and Benjamin.