She was the daughter of Thomas Randolph, 1st Earl of Moray, nephew and companion-in-arms of Robert the Bruce, and Moray's wife, Isabel Stewart, herself a daughter of John Stewart of Bonkyll. Agnes became renowned for her heroic defence of Dunbar Castle against an English attack by the William Montagu, 1st Earl of Salisbury, which began on 13 January 1338 and ended on June 10, 1338.