Ebersol was born in Torrington, Connecticut, the son of Mary (née Duncan) and Charles Roberts Ebersol, a former chairman of the American Cancer Society. Beginning in 1967, at the age of twenty, Ebersol began his long history with the Olympics when he temporarily dropped out of Yale University to join Roone Arledge and ABC Sports as television’s first-ever Olympic researcher.