Gallo was born on January 8, 1912 in Alabama, but grew up in the Little Italy section of Manhattan, New York. Gallo was married and was the father of Gambino associate Joseph C. Gallo. Gallo and his family lived in Long Island City, Queens. In the 1930's, Gallo was convicted in New York of illegal gambling. Over the years, Gallo built his power base in the New York garment industry. He owned a dress manufacturing company in Brooklyn and eventually controlled the Greater Blouse, Shirt, and Undergarment Association, a trade group. Gallo also had strong ties with the Trafficante crime family of Tampa, Florida and the New Orleans crime family under boss Carlos Marcello. Gallo frequently represented their leaders at Cosa Nostra meetings in New York.