The tower-fortress was built in the second half of the 14th century of local limestone. Tower-strongholds were built by vassals to protect roads and waterways and to protect themselves against peasant uprisings. Construction of such tower-strongholds increased after the failed St. George's Night uprising by peasants in 1343.
In 1986, the fortress was restored under the leadership of Vao sovkhoz. Exhibition on the I floor introduces the history of Vao Tower-Fortress and Manor, and the surrounding villages. The II floor is a medieval dwelling that includes the washing area (place for washing hands), and dansker (dry toilet). The current furnishing imitates the pristine appearance of the room. The windows are of beautiful stained glass. The staircase in the wall leading to the cellar adds mysticism to the place. The III floor is enlivened by paintings created by the artist E.Veermäe which depict people in medieval clothing.
Ängsö Castle was first named as "Engsev" in a royal charter by king Canute I of Sweden (r. 1167-1196), in which he stated that he had inherited the property after his father Eric IX of Sweden. Until 1272, it was owned by the Riseberga Abbey, and then taken over by Gregers Birgersson.
From 1475 until 1710, it was owned by the Sparre family. The current castle was built as a fortress by riksråd Bengt Fadersson Sparre in the 1480s. In 1522, Ängsö Castle was taken after a siege by king Gustav Vasa, since its owner, Fadersson's son Knut Bengtsson, sided with Christian II of Denmark. However, in 1538 it was given by the king to Bengtsson's daughter Hillevi Knutsdotter, who was married to Arvid Trolle.
In 1710, the castle was taken over by Carl Piper and Christina Piper. Ängsö Castle was owned by the Piper family from 1710 until 1971, and is now owned by the Westmanna foundation. The castle building itself was made into a museum in 1959 and was made a listed building in 1965. It is currently opened to visitors during the summers.
The castle is a cubical building in four stores made by stone and bricks. The lower parts is preserved from the middle ages. It was redecorated and expanded in the 1630s. The 4th storey as well as the roof is from the expansion of Carl Hårleman from 1740-41. It gained its current appearance in the 1740s.