He was the son of William Gordon, 2nd Earl of Aberdeen and his third wife Lady Anne Gordon. Educated at Edinburgh University, he was commissioned a cornet in the 11th Regiment of Dragoons in 1756. On 11 August 1759, he was appointed to a captaincy in the newly raised 16th Regiment of (Light) Dragoons. Appointed a lieutenant-colonel in the 105th Regiment of Foot (Queen's Own Royal Regiment of Highlanders) on 11 October 1762, he went on half-pay when that regiment was disbanded in 1763.