The oldest part of the castle in Olsztyn is a brick cylindrical tower standing at the highest point of the hill. It comes from the thirteenth century. Do not preserved, unfortunately, documents evidencing the construction of the castle, on the basis of the study can be said that only Casimir the Great in the mid-fourteenth century increased the size of the defense establishment largest stronghold on the Polish Jura Chain. The extent of expansion may indicate the change of names. While in previous documents from the process of Bishop John Muskata brought by Archbishop James Mumps speaks of "the castle Przymiłowice ', that after the extension made ​​by Casimir the Great, the castle is called" Holstein "or" Olstin ". Surely these are the same place, but not exactly the same object.

The earliest mention of the existence indirectly in Olsztyn castle comes from 1349 years. The document was witnessed zdzisko margrave de Olsten. March 15, 1369, King Kazimierz the Great stayed at the castle, when the law gave Środa for nearby Przyrów. This document is preserved in its entirety to our times, but most likely he was already king of the castle many times.

Castle, in the form, the ruins of which still exist today, consists of three parts. Upper castle is the oldest, as already mentioned at the beginning of the tower dates back to the thirteenth century. There are 35 feet high, the base is round and built of stone. In the upper part extended with her ​​in the fifteenth century ośmiobocznie brick. Was the site of the final defense, the entrance to her lead from the outer walls. Dungeons were used to hold people sentenced to death by starvation. On the upper castle is a large cavern, probably served for storage of food and ammunition. Passageways and corridors carved into the rock formed together with natural caves, a labyrinth throughout the castle.

From the middle of the castle survived only a few traces, like the lower castle. In the bottom of the castle were utility rooms, stables and living quarters for servants and ancillary buildings. In addition to these three components constituting a unified whole on the opposite side of the hill stood a square tower alone. Served observation functions, as well as being able to flank the defense access to the walls of the castle proper. The castle was manned by a permanent crew of King Casimir the military.

After the expiry of the Piast dynasty castle got together with several other nearby castles in the hands of Ladislaus of Opole. He was a robber and conspired with the Teutonic Knights, only Wladyslaw Jagiello curtail his actions. He restored the crown of locks that are in the hands of Opole, including after a week siege in 1396, the castle Olsztyn, again manning its permanent crew.

In 1587 was fought the battle of Olsztyn castle, which later wrote Aleksander Fredro and Wladyslaw Syrokomla. For the Polish troops entered the Archduke Maximilian, on their way stood the castle of Olsztyn, whose crew commanded by Kacper Karliński. The Austrians went so far as to kidnap the son of Polish commander and keeping it in the first line of the advancing. Tradition has it that none of the crew dared to burn the enemy, then Karliński he fired the first shot. The battle killed his son, but the army had to withdraw the prince.