Lalla is the daughter of Edward Ward, the 7th Viscount Bangor, and his fourth wife, Marjorie Alice Banks. She has a brother, Edward Nicholas Ward, two years her junior, and a half-brother, William Maxwell David Ward, who is three years her senior and the 8th Viscount Bangor. Lalla Ward is herself more properly addressed as the Honourable Sarah Ward. She is descended from George Plantagenet, 1st Duke of Clarence, brother of Edward IV, via John Mordaunt, 1st Earl of Peterborough, John, 1st Viscount Mordaunt, and Bernard Ward, 1st Viscount Bangor.