The son of his father and mother, Edwards played college football at the University of California in 1972 and 1974, at Monterey Peninsula Junior College in 1973, and at San Diego State in his senior year, 1975. Classmates say he showed a distinct lack of interest in anything but football, and showed none of his future skills. In fact a common joke was "Herm could burn water".