He was the son of Gabriel, comte de Lauzun, and his wife Charlotte, daughter of Henri-Nompar de Caumont, duc de La Force. He was brought up with the children of his relative, the maréchal de Gramont. One of them, Armand de Gramont, comte de Guiche, became the lover of Henrietta Anne Stuart, Duchess of Orléans, while a daughter, Catherine Charlotte, afterwards princess of Monaco by marriage to Louis I Grimaldi, was the object of the one passion of Lauzun's life. See the letters of Madame de Sévigné and the memoirs of Saint-Simon, who was Lauzun's wife's brother-in-law.