D'Orvilliers was born in Moulins, Allier, but spent most of his childhood in Cayenne, capital of the French colony French Guiana, where his father was governor. In 1723, aged fifteen, he joined the colony's infantry regiment and quickly rose to the rank of Lieutenant. In 1728, he transferred to the Navy and, by 1756, had become a captain, commanding one of the ships sent to Minorca under the direction of La Galissonière. He later took part in action near Santo Domingo and the Antilles and was rewarded with a promotion to Vice Admiral in 1764.