Carroll was born to a working-class family of Irish descent, and grew up on New York's Lower East Side, and when he was about eleven (in the sixth grade) his family moved north to Inwood in Upper Manhattan where he attended Good Shepherd School.He was taught by the LaSalle Christian Brothers, and his brother in the sixth grade noted that he could write and encouraged him to do so. In fall 1963, he entered public school, but was soon awarded a scholarship to the elite Trinity School. He attended Trinity from 1964-1968. He briefly attended Wagner College and Columbia University.