Milano was born in 1925 in Cleveland, Ohio to Anthony and Josephine Milano. His father was Underboss of the Cleveland crime family from the 1930s until his retirement in 1976. Many members of the Milano family were involved in organized crime, including his uncle Frank Milano, who was Boss of the Cleveland crime family and sat on the original Commission from 1931 until he chose to flee the country to avoid tax evasion charges in 1935. His brothers were also involved in the Mafia except for Carmen Milano who chose to be a lawyer until eventually also becoming a mobster in the 1980s. He was also related through marriage to John Nardi who was a Mafia associate who was killed for switching sides to the Irish Mob, during the Cleveland family's war with Danny Greene. Milano moved with his family