As a quarterback for the Miami Hurricanes, James set five school passing records. He then received a master's degree in education from the University of Kansas, where he was a graduate assistant for the Jayhawks. James coached high school football at Southwest Miami High School in 1959, then was a college assistant coach for 12 seasons at Florida State University, the University of Michigan, and the University of Colorado at Boulder.