The castle was probably built in the 13th century most likely in place of an ancient Estonian stronghold. The bailiwick of Karksi was first mentioned in 1248. The stronghold had a chapel dedicated to Apostle Peter. The first reference was made to a local clergyman in 1298. The present stone church, very simple in design, was built in the same place between 1773 and 1778. St. Peter’s Church is in the ruins of Karksi castle. The tower of the church is leaning, the inclination of the tower’s top is 205 cm at the moment. In 1994 it was decided to save the tower using complicated "construction surgery" developed while stabilizing structures in the old town of Tartu. Eight 10- metre-tall bored piles along with concrete beams penetrating foundations form a new bearing surface for the tower. As a result, the further leaning of the tower has stopped.
On the way which takes you to the stronghold, in the former cemetery of the Knights of the Order of the Sword, stands a nice Baroque chapel among trees, built in the early 18th century by Field Marshal George Reinhold von Lieven, the owner of the Karksi manorial estate. The coats of arms of the Lievens and the Mannteuffels have been fixed to the chapel doors. The chapel, similarly, was built with one side on the former foundation and therefore is somewhat leaning. The buildings of the Karksi manor have perished.
St. Stephen"s Basilica is a Roman Catholic basilica named in honour of Stephen, the first King of Hungary (c. 975–1038), whose supposed right hand is housed in the reliquary. It was the sixth largest church building in Hungary before 1920. Today, it is the third largest church building in present-day Hungary.
The basilica was completed in 1905 after 54 years of construction, according to the plans of Miklós Ybl, and was completed by József Kauser. Much of this delay can be attributed to the collapse of the dome in 1868 which required complete demolition of the completed works and rebuilding from the ground up.
The architectural style is Neo-Classical; it has a Greek cross ground plan. The façade is anchored by two large bell towers. In the southern tower is Hungary"s biggest bell, weighing over 9 tonnes. Its predecessor had a weight of almost 8 tonnes, but it was used for military purposes during World War II. Visitors may access the dome by elevators or by climbing 364 stairs for a 360° view overlooking Budapest.
At first, the building was supposed to be named after Saint Leopold, the patron saint of Austria, but the plan was changed in the very last minute, so it became St. Stephen"s Basilica.
The Saint Stephen Basilica has played an active role in the musical community since its consecration in 1905. The head organists of the church have always been very highly regarded musicians. In the past century the Basilica has been home to choral music, classical music as well as contemporary musical performances. The Basilica choir performs often in different parts of Europe as well as at home. In the summer months they perform every Sunday. During these months you can see performances from many distinguished Hungarian and foreign organ players alike.