Born in Hoosick Falls, New York, Haynes moved to Ohio with his parents, who settled in Lower Sandusky (now Fremont) in 1839. He attended the common schools. Apprenticed as a printer. He worked at the Sandusky Clarion and the Cleveland Plain Dealer. He served as clerk on a steamer on Lake Superior in 1848 and 1849. He engaged in mercantile pursuits at Fremont 1850-1856. Auditor of Sandusky County, Ohio from 1856 to 1860. Enlisted in the Union Army as a private April 16, 1861, in the Eighth Regiment, Ohio Volunteer Infantry. Commissioned captain and served in western Virginia, the Shenandoah Valley, and in the Army of the Potomac until November 1862, when he was commissioned lieutenant colonel of the Tenth Regiment, Ohio Volunteer Cavalry, and served with it in the Army of the Cumberland unt