Ajman castle was built by the Loka chief Matija Lampfrizheim in the 17th century (in the year 1689) on the land of three farms.
A large number of huts belonged to the castle.
Its name remained from the landlord Dr. Jože Hayman, who acquired it in the 18th century (in the year 1791).
In the year 1847, it came into the hands of the Detela family by marriage; one member of this family was the county chieftain Oton Detela.
Before World War II it was the property of Dr. Jernej Demšar, a specialist in dermatology in Ljubljana.
During the occupation it was damaged in a fire.
In the year 1961 the Loka Ursuline order moved into the renovated castle.