Born in New York City, Groh is a native of Manhasset, New York, on the North Shore of Long Island. He is a 1962 graduate of Chaminade High School, where he played on Chaminade's undefeated, untied 1961 football team under Coach Joe Thomas. He is also a 1967 graduate of the University of Virginia’s McIntire School of Commerce. He returned to his alma mater in 2001 as the head coach of the Cavaliers football team. His son Mike Groh was a high school football star at Randolph High School in Randolph, New Jersey who kicked the game winning field goal in The Star-Ledger's "Greatest High School Football Game Ever Played" in 1990. Mike was formerly his father's offensive coordinator, quarterback coach, and recruiting coordinator for the Cavaliers.