The castle was probably built in the 16th century by a member of the Savage family. In 1635, Patrick Savage's brother-in-law, Sir James Montgomery of Rosemount (Greyabbey), repaired the castle by roofing and flooring it so that his sister could live in greater comfort there.