This two-storey Renaissance castle building stands at the edge of a slope above the densely built city of Idrija. The whole thing was constructed on the order of the Hapsburgs as a mine storehouse and mine administration facility in 1527, a decade after the devastating Idrija earthquake. Around 1735 the castle was rebuilt in Baroque style.

The castle is comprised of four wings, which surround a large inner courtyard. The protruding corners of the building are reinforced with cylindrical defence towers on the outside. The crowning clock tower from a later period faces the town. The inner yard is arcaded by several wings and on all three floors. Between the arches are reconstructed, roughly painted, lively decorations. The arcades were glazed for easier use and the yard was paved. Part of the inner areas with remains of the building fittings has been preserved. Since 1953 the castle has been renovated into a local museum, and the attic into the hall of a music school (architect M. Suhadolc).

The museum, with various collections with an emphasis on the development of the mine, with memorial rooms dedicated to the writer France Bevk and politician Aleš Bebler, and with a special lace-making collection, is open regularly.