Born at Great Marlow, in the county of Buckinghamshire, and christened James Robert Hope, he was the third son of Sir Alexander Hope, and grandson of John Hope, 2nd Earl of Hopetoun. After a childhood spent at the Royal Military College, Sandhurst, of which his father was commander, he was educated at Eton College and Christ Church, Oxford, where he was a contemporary and friend of William Ewart Gladstone and John Henry Newman. In 1838 he was called to the bar at Lincoln's Inn. Between 1840 and 1843 he helped to found Trinity College, Glenalmond. In 1840-41 he spent some eight months in Italy, Rome included, in company with his close friend Edward Louth Badeley.