Altensteig Castle probably dates back to the 12th century. In 1287, the castle was first mentioned as being owned by the Counts of Hohenberg. Hermann, son of Margrave Rudolf of Baden, conquered the town but had to return it to Count Burkhard von Hohenberg in 1288. In 1398, the Hohenbergers sold the Lordship of Altensteig to the Margraves of Baden. The castle became the seat of the Baden bailiffs.
In 1603, as part of a territorial exchange, Altensteig came under the rule of the Duchy of Württemberg. From 1605, Heinrich Schickhardt built the New Castle as the administrative and residential seat of the Württemberg bailiffs. The old castle served as a grain silo from the 18th century onwards. The lower part of the striking residential tower, with a floor area of about 11 meters by 16 meters, is made of sandstone masonry. The construction suggests that it was built in the 12th and 13th centuries. Presumably, an Alemannic timber-framed building was added to it in the 15th century. The six floors housed cellars, the great hall, kitchen, living quarters, and servant quarters. The largest room, now called the 'Knight's Hall,' is located on the second floor.
On the north side, the castle complex was protected by a wall with a battlement and two towers. The towers once served as a prison. While the western tower, called the 'Heaven' tower, had barred windows that looked out to the outside, the eastern tower, called the 'Hell' tower, had a 15-meter deep dungeon. The fortifications were connected to the city wall.
Dryburgh Abbey on the banks of the River Tweed in the Scottish Borders was founded in 1150 in an agreement between Hugh de Morville, Constable of Scotland, and the Premonstratensian canons regular from Alnwick Abbey in Northumberland. The arrival of the canons along with their first abbot, Roger, took place in 1152.
It was burned by English troops in 1322, after which it was restored only to be again burned by Richard II in 1385, but it flourished in the fifteenth century. It was finally destroyed in 1544, briefly surviving until the Scottish Reformation, when it was given to the Earl of Mar by James VI of Scotland. It is now a designated scheduled monument and the surrounding landscape is included in the Inventory of Gardens and Designed Landscapes in Scotland.
David Erskine, 11th Earl of Buchan bought the land in 1786. Sir Walter Scott and Douglas Haig are buried in its grounds.