After graduating from Benedictine High School in 1955, Ross enrolled at the Virginia Military Institute, where he started at quarterback and defensive back for two seasons and served as captain of the football team as a senior. Ross graduated from VMI in 1959 with a bachelor of arts degree in English and history.