Richards was born in 1874 in Camden, Alabama. He was the only son of De Forest Richards. His father was a New Hampshire native who moved to Alabama in the Reconstruction era, serving as the sheriff of Wilcox County, Alabama, and operating a tannery. Branded as a "carpetbagger", his father moved to Nebraska when Richards was 12 years old. Richards attended public school in Nebraska for two years before being sent to the St. Paul's School in Concord, New Hampshire. After graduating from the St. Paul's School in 1892, Richards moved to Casper, Wyoming, where his father had established a store under the name Richards, Cunningham & Co.