Kelly was born in Everett, Massachusetts, and was raised in an Irish American Catholic family in Chelsea, Massachusetts. He attended St. John's Preparatory School in Danvers, Massachusetts. His father was a Boston politician. He was a four-year letter winner at Assumption College as a linebacker. After graduating from Assumption in 1983 with a bachelor's degree in political science he served as linebackers coach, defensive coordinator, and softball coach from 1983 - 1986.