The first castle was built on the high hill named Várhegy at Felsőtárkány near Eger.

During the Mongol invasion in 1241, this castle was ruined, and the bishop of Eger moved it to a rocky hill in the city of Eger. On the hill, a new castle was built, and it developed rapidly. In 1470 a Gothic palace was built. In 1552, a Turkish army of 35,000-40,000 soldiers attacked the castle which had 2,100-2,300 defenders. The siege failed as the Turks suffered heavy casualties. A total of 1,700 of the defenders survived. After that Turks besieged the castle again in 1596, resulting in a Turkish victory. In 1701, the Austrians exploded half of the castle (the Külső vár).

Archaeological excavations only started in 1925 and the castle was used by the army as barracks until 1957.