Shelley was born in Sullivan County, Tennessee. He moved with his father to Selma, Alabama, in 1836. He received limited schooling, but showed an aptitude for architecture. He became an architect and builder in the 1850s. In the 1860, 1880, and 1900 Federal Censuses, he was listed as having been born in Tennessee (as were his parents in the latter two censuses).