Smith was born in Pimlico, Middlesex (now London), England the daughter of an American merchant from Brooklyn, Charles Edward Smith and his American wife Corinne Colman. Due to her father’s job with the West India Improvement Company, the family often moved, spending time in London, Kingston, Jamaica and Brooklyn, New York.