He was born in Cingoli, Marche, the son of Count Ottavio Castiglioni and his wife Sanzia Ghislieri. He studied Canon law and, in 1800 became bishop of Montalto. After he refused to swear allegiance to Napoleon I of France (1804–14, 1815) he was taken to France, but following the defeat of France, he was, in 1816, made a Cardinal-Priest of Santa Maria in Traspontina. He held various high offices thereafter, including that of Major Penitentiary.