The castle was probably built in the 13th century most likely in place of an ancient Estonian stronghold. The bailiwick of Karksi was first mentioned in 1248. The stronghold had a chapel dedicated to Apostle Peter. The first reference was made to a local clergyman in 1298. The present stone church, very simple in design, was built in the same place between 1773 and 1778. St. Peter’s Church is in the ruins of Karksi castle. The tower of the church is leaning, the inclination of the tower’s top is 205 cm at the moment. In 1994 it was decided to save the tower using complicated "construction surgery" developed while stabilizing structures in the old town of Tartu. Eight 10- metre-tall bored piles along with concrete beams penetrating foundations form a new bearing surface for the tower. As a result, the further leaning of the tower has stopped.
On the way which takes you to the stronghold, in the former cemetery of the Knights of the Order of the Sword, stands a nice Baroque chapel among trees, built in the early 18th century by Field Marshal George Reinhold von Lieven, the owner of the Karksi manorial estate. The coats of arms of the Lievens and the Mannteuffels have been fixed to the chapel doors. The chapel, similarly, was built with one side on the former foundation and therefore is somewhat leaning. The buildings of the Karksi manor have perished.
Visby Cathedral (also known as St. Mary’s Church) is the only survived medieval church in Visby. It was originally built for German merchants and inaugurated in 1225. Around the year 1350 the church was enlarged and converted into a basilica. The two-storey magazine was also added then above the nave as a warehouse for merchants.
Following the Reformation, the church was transformed into a parish church for the town of Visby. All other churches were abandoned. Shortly after the Reformation, in 1572, Gotland was made into its own Diocese, and the church designated its cathedral.
There is not much left of the original interior. The font is made of local red marble in the 13th century. The pulpit was made in Lübeck in 1684. There are 400 graves under the church floor.