The castle was first mentioned in written sources in 1274. The castle fell into ruin by the 15th century or in the earthquake of 1511. In 1635, PrinceJohann Anton Eggenberg purchased the Stegberg Estate and built Lower Stegberg Manor from the ruins of the old castle on the site of a former manor and the later castle farm. Count Johann Caspar Cobenzl purchased the manor and the estate from the Eggenbergs in 1761. In 1846, Prince Weriand of Windischgrätz became the owner. The property remained in the possession of his heirs until the end of the Second World War.