Ruttenstein Castle was first time documented in 1209, in a deed issued by Duke Leopold for the Baumgartenberg Monastery, where it is mentioned under the name Rotenstein. The existing structure of the high castle, however, suggests that the castle is significantly older. The construction period likely falls in the first half of the 12th century, although this is not documented. It is probably a foundation of the Counts of Clam-Velburg.
The walls of the formerly massive pentagonal keep are now only partially preserved, tapering to a point and visibly towering over the site. The former residential wing in the west retains three of its walls. Above them, a viewing platform has been constructed, accessible via wooden stairs. Clearly recognizable, though also only partially preserved, is the early Gothic castle chapel located near the stone steps to the residential building. The main castle itself was protected on the north, south, and east by a mighty ring wall, which is still partially preserved today, and from which seven wall towers protruded.
Dryburgh Abbey on the banks of the River Tweed in the Scottish Borders was founded in 1150 in an agreement between Hugh de Morville, Constable of Scotland, and the Premonstratensian canons regular from Alnwick Abbey in Northumberland. The arrival of the canons along with their first abbot, Roger, took place in 1152.
It was burned by English troops in 1322, after which it was restored only to be again burned by Richard II in 1385, but it flourished in the fifteenth century. It was finally destroyed in 1544, briefly surviving until the Scottish Reformation, when it was given to the Earl of Mar by James VI of Scotland. It is now a designated scheduled monument and the surrounding landscape is included in the Inventory of Gardens and Designed Landscapes in Scotland.
David Erskine, 11th Earl of Buchan bought the land in 1786. Sir Walter Scott and Douglas Haig are buried in its grounds.