Eliott was born at Wells House, near Stobs Castle, Roxburghshire, the 10th (and 8th surviving) son of Sir Gilbert Eliott, 3rd Baronet of Stobs, by his distant cousin Eleanor, daughter of William Elliot of Wells, also in Roxburghshire. His aunt Charlotte Elliot, sister of Eleanor, had married Roger Elliott, another Governor of Gibraltar.