Château de Dieppe was founded in 1188 and destroyed in 1195. The site was restored in the 14th century. The castle was largely reconstructed by Charles des Marets in 1433. The castle is composed of a quadrangular enclosure with round flanking towers and a lower court adjacent. The large west tower dates perhaps from the 14th century, and served as the keep. Several architectural styles are represented, and flint and sandstone are used in the buildings. A brick bastion and various other buildings have been added to the original enclosure. The town walls were built around 1360. The walls were extended between 1435 and 1442. Although the town was largely destroyed by an Anglo-Dutch naval bombardment in 1694, the castle survived.
Until 1923, the castle housed the Ruffin barracks. It was bought by the town in 1903 and today is home to the Dieppe museum with its collection of ivories (crucifixes, rosaries, statuettes, fans, snuffboxes, etc.), maritime exhibits and the papers and belongings of Camille Saint-Saëns. The castle offers a panoramic view over the town and the coast.