Carneglia was born in Brooklyn, New York to Italian immigrants. At an early age Carneglia showed himself as a rebellious teenager and started to work for the Gambino crime family. In 1969, Carneglia, his older brother John Carneglia and Angelo Ruggiero were arrested for hijacking three trucks in New Jersey. The trucks were carrying $120,000 worth of merchandise.