Richardson was born on 20 March 1861 to Oliver P. and Hester F. (Wingo) Richardson in Hunt County, Texas. After attending schools in that county, the younger Richardson received an appointment to attend the United States Military Academy at West Point in 1880. After graduation in 1884, Richardson was commissioned as a second lieutenant in the 8th Cavalry Regiment, serving in California and other western posts. Richardson was promoted to first lieutenant in 1889, and joined the faculty at West Point three years later.