Ernest was a son of Duke Albert V of Bavaria. He was educated and trained by the Jesuits. He was elected the Bishop of Freising in 1566 at age 12 and the Bishop of Hildesheim in 1573 at age 19. In 1577 he was not elected the Archbishop of Cologne, with Gebhard of Truchsess Waldburg defeating him 12 votes to 10. In 1581 he was elected the Bishop of Liège.