Cadwallon ap Madog was the son of Madog ab Idnerth who had died in 1140, while Idnerth was a grandson of Elystan Glodrydd who had died in around 1010 and had founded a dynasty in the Middle Marches of Wales, in the area known as Rhwng Gwy a Hafren (Between Wye & Severn).