The village of Grillenberg was mentioned as early as 880/890 in the Hersfeld tithe register (Hersfelder Zehntverzeichnis). At that time there is no indication of a castle, however. The latter was first recorded when, in 1217, a certain Tidericus de Grellenberch is named as a vassal of the Archbishopric of Magdeburg. The lower parts of the walls of the ruin, made of large rusticated ashlars, must date back to the original Romanesque fortification. In 1286 the marshals (Burgmannen) from the Muser and von Morungen families are named.