Mivart was born in London. His parents were Evangelicals, and his father was the wealthy owner of Mivart's Hotel (now Claridge's). His education started at the Clapham Grammar School, and continued at Harrow School and King's College London. Later he was instructed at St Mary's, Oscott (1844–1846); he was confirmed there on 11 May 1845. His conversion to Roman Catholicism automatically excluded him from the University of Oxford, then open only to members of the Anglican faith.