Baldwin was born in Massapequa, New York, the son of Carol Newcomb (née Martineau) and Alexander Rae Baldwin, Jr. His father was a high school history/social studies teacher and football coach. He is the brother of actors Alec, Daniel, and Stephen, sometimes collectively known as the Baldwin brothers, and of sisters Beth and Jane, all together known as the Baldwin family. Baldwin was raised in a Catholic family, and has English, Irish, Scottish, French-Canadian, and German ancestry. A graduate of Alfred G. Berner High School and Binghamton University, where he was a varsity wrestler, he has a degree in political science.