Ellerson attended Salpointe Catholic High School in Tucson, Arizona. On 28 September 2002, he was inducted into the Salpointe Catholic High School Sports Hall of Fame. He received a congressional appointment to the United States Naval Academy, which he attended for his freshmen year of 1972–73. He subsequently attended the University of Hawaii between 1974 and 1977 where he played for the Hawaii football team as a center and linebacker.