The present castle probably replaces a previous castle on the site. It is thought that the castle was built for the marriage of Alexander Fraser and Margaret Abernethy of Saltoun in 1596. This marriage led to the Frasers inheriting the Saltoun peerage. The Cumines acquired and enlarged the property. It was purchased by Sir William Forbes towards the end of the 18th century, but subsequently abandoned.