Celje's first defensive building, known as the Old Castle, was built in the first half of the 13C. A century later, it passed to the lords of Žovnek, Counts of Celje after 1341. Their ambitious reconstruction, maintaining its defensive strength, made it a prestigious residence. This is evidenced by the Frederick Tower, which dominates the eastern part of the castle. After several modifications, it ended up being used as a quarry in the 19C; then, as of 1882, the powerful fortification began to reemerge from its ruins.