The grandson of Finnish immigrant builders, Eldred was raised in Minnesota. Eldred moved to Kansas City in 1959, fresh out of the University of Michigan. Within a year, he was named chairman of the sculpture department of Kansas City Art Institute. Eldred possessed an imposing physical presence and was a college football fullback. He was known to be resilient in the face of challenge, such as the fire in 1991 that destroyed a studio that contained his library and many valuable artworks.