Born in Brooklyn, New York to Lawrence and Maria Tierney, and nicknamed "Jerry", Brady's father was chief of the New York aqueduct police force. His older and younger brothers were fellow actors Lawrence and Edward Tierney, respectively. Edward subsequently left acting to become a building contractor, and Scott financed some of his projects.