The first incarnation of the fort was built between 1652-3 during the First Anglo-Dutch War. Little is known of the original structure and no trace of it has been found. The second structure was built on the same site by Robert Mylne under the orders of Charles II at the start of the Second Anglo-Dutch War in 1665 at a cost of £28,000. It held off a Dutch fleet in 1667 which thought it was far more heavily manned and gunned than it actually was. In fact, the walls were unfinished and there were few guns. At the end of the war it was slighted when the government decided not to station a garrison in Lerwick, and it was unmanned when the Dutch burnt it in 1673 during the Third Anglo-Dutch War.