Toledo played football at Lincoln High School in San Jose, California. He was the starting quarterback from 1961 to 1963. Toledo played for San Jose State during the 1964 season. In 1965, Toledo transferred to San Jose City College, where he was the starting quarterback and was a junior college All-American. Toledo was the starting quarterback for the San Francisco State Gators during the 1966 and 1967 seasons. While at SFSU, the team went 16–5 and played in the 1967 Camellia Bowl, a defeat against Don Coryell's San Diego State Aztecs. Toledo graduated from San Francisco State in 1968 and then tried out for the San Francisco 49ers.