When he was young, Williams' family moved to Detroit, and he was 15 when he dropped out of school to work as an apprentice machinist in Ohio, soon relocating to Arkansas and Oklahoma, where he drifted about, sometimes working on ranches during a six-year period. He spent three years in the U.S. Cavalry.