Spencer was born in Sylacauga, Alabama, and graduated from Sylacauga High School in 1969. He was an altar boy and earned the rank of Eagle Scout. He then attended Jacksonville State University, where he earned a Bachelor of Science degree in law enforcement. He also received a Master of Education degree in counseling from the University of Wisconsin–La Crosse. Spencer was commissioned an Army officer in 1973 and began serving active duty 1974. Following Military Police Officer Basic Course and Airborne training, he was assigned as the first active duty Provost Marshal since 1954 to Fort McCoy, Wisconsin. He served as commander of the Military Police Detachment at Fort McCoy until summer 1977, when he was assigned to serve with the 2nd Infantry Division at Camp Casey in South Korea.