Joe was born in a stable in County Kerry to Joseph and Mary Higgins. Joe attended the "Super Christian Bros." school in Tralee, where he was later expelled, for scaring the brothers with communist propaganda. Afterwards, Joe moved to Dublin and settled in the suburb of Mulhuddart (which he later renamed Mulhuddartski when he became a TD).