Raffalovichs was born into a wealthy Jewish family, which moved from Odessa to Paris in 1863. He went up to study in Oxford in 1882 before settling down in London and opening a salon in the 1890s. Oscar Wilde attended, calling the event a saloon rather than a salon. This is where Raffalovich met the love and companion of his life, John Gray. In 1890 his sister Sophie married the Irish nationalist politician William O'Brien (1852-1928).