Wolfe was born in Philadelphia and raised in Madison, Wisconsin. He currently resides in Los Angeles. Wolfe graduated from Emory University in 1994. He later earned his Masters degree at USC. As a student Wolfe did both play-by-play and color commentary for the Trojans' basketball, football, and baseball teams on KSCR (104.7 FM) in Los Angeles. He also hosted the sports talk show, Blackjack and the Wolfe Attack.