Rains was born on April 10, 1833 in Nashville, Tennessee, the son of Methodist minister Rev. John and Lucinda Cartwright Rains. He spent his youth making tack in his father's small saddlery. A benefactor lent him $400 to attend Yale, where he graduated second in the Class of 1854 at Yale Law School. He served as headmaster at the Millwood Institute in newly formed Cheatham County. He was associate editor of the Daily Republican Banner, serving under a future fellow Confederate general, Felix Zollicoffer. Rains was an active member of the Whig Party, and was initially opposed to secession. He was elected the Nashville city attorney in 1858.