Von Rosen was born in Helgesta, Flen Municipality, Södermanland, Sweden son of the explorer Eric von Rosen (1879–1948) and nephew of Carin Göring, wife of Hermann Göring. Throughout his upbringing, Carl Gustaf appeared to have had serious differences in opinion with his family, with one author referring to him as the "black sheep of his family."