Moyenmoutier abbey was founded by Saint Hydulphe in 671. In 915 the abbey was pillaged and torched by a Hungarian raiding party; it was rebuilt around 960. In the 12th century Aubert of Paroye built a hilltop castle on the Rock of St Peter, but this was destroyed in the 13th century under orders from the Duke of Lorraine.
The monastery dominated local life, sharing the ups and downs of the ensuing centuries with the little town that surrounded it. By the end of the 16th century, with the religious world across western Europe undergoing the confrontation between the Protestant winds from Germany and the Counter Reformation forces confronting it, there was a feeling that monasticism in Moyenmoutier had fallen into a sorry state. The commendatory abbot of Moyenmoutier, Eric of Lorraine who was also the Bishop of Verdun and commendatory abbot of the Monastery of St Vanne in Verdun, undertook to restore a more orderly existence to the monks under his authority. He entrusted the task to Dom Didier de La Cour. In 1604 the resulting new constitution of the Congrégation of St. Vanne and St. Hydulphe received papal approval. The reforms became an important element in the Counter Reformation, being copied by other Catholic religious houses and congregations including in 1621, most famously, the Congregation of St. Maur.
The 18th century was a golden age for the Abbey at Moyenmoutier which for many years numbered among its scholars of philosophy and theology Dom Augustin Calmet, who in 1728 went on to become abbot at Senones. The French Revolution called time on the monastic existence, however, and in 19th century Moyenmoutier the chanting of monks was replaced by noise of industrial machines.
The elegant simplicity of Moyenmoutier church, which has the dimensions of a cathedral, contrasts with its exceptional furniture, in particular the organ and 17th century choir stalls.
Celje Castle was once the largest fortification on Slovenian territory. The first fortified building on the site (a Romanesque palace) was built in the first half of the 13th century by the Counts of Heunburg from Carinthia on the stony outcrop on the western side of the ridge where the castle stands. It had five sides, or four plus the southern side, which was a natural defence. The first written records of the castle date back to between 1125 and 1137; it was probably built by Count Gunter. In the western section of the castle, there was a building with several floors. Remains of the walls of this palatium have survived. In the eastern section, there was an enclosed courtyard with large water reservoirs. The eastern wall, which protects the castle from its most exposed side, was around three metres thicker than the rest of the curtain wall. The wall was topped with a parapet and protected walkway.