He played junior hockey for the Toronto Marlboros from 1951 to 1955 and won the Memorial Cup in 1955. He then turned pro with the Toronto Maple Leafs through the 1959-60 season, all the time also playing football. He missed the 1958-59 season because of a football injury. In the NHL he played in 149 games and scored 14 goals and 26 assists. He played one last season of pro hockey with the Winnipeg Warriors in 1960-61. James retired from football in 1964. He then joined the senior Yorkton Terriers for three seasons, retiring in 1967. He made a one-year comeback in 1971-72.