
The castle was probably founded by a monarch, although no documents exist which describe the beginnings of its history. The first written records on Zbořený Kostelec (originally named Kostelec nad Sázavou) date from as late as 1341 when King John of Luxembourg handed it to Oldřich of Valdek, owner of neighbouring Týnec, as security. The new proprietor evidently did not treat his new serfs well, however, as testified by their complaints to the king. The justification of their demands are borne out in a charter in which the future King Charles IV., in the absence of John of Luxembourg, granted that the inhabitants of the town of Vadslavice and six other villages belonging to the estate, could buy their freedom. Charles IV later bought Kostelec castle itself and again relinquished it several times as security (among other things, Kostelec is included in the Maiestas Carolina among those royal seats which could be released as security for a maximum period of six years and only "for diligent use and the common good of the Czech Land"). The castle was entrusted to Jan of Hardek, the burgrave of Magdeburg, in 1360. After the death of Charles IV. the castle was returned to the Crown for a while and it was later relinquished as security to the Vyšehrad burgrave Chval of Rzavý who finally acquired hereditary custody of the estate. After 1399 the castle was held by Filip Laut of Dědice, the highest master of the hunt in the Czech Kingdom.
In 1411 the castle was transferred to the hands of the distinguished Jan Sádlo of Smilkov, adviser and favourite of King Wenceslas IV. In view of his position he also maintained high offices - from 1407 he was the royal chamberlain, later, for example, he became the burgrave at Karlštejn castle and a member of the royal council. Jan was a moderate supporter of the Hussite movement, thus in time he was suspected of treason. When he learned of this suspicion he tried to clear his name and sent a request for a letter of safe-conduct to Prague in order that he could travel to the city. The Praguers wrote him a document stating that he could journey in safety and they did not believe the rumours that he was an informer. However, he arrived in Prague in the company of other noblemen on Monday 20 October 1421 "and when he entered the town hall with the above-mentioned lords at approximately the hour of vespers in order to justify his actions, the aldermen ordered his arrest and at two o'clock in the morning that same day he was beheaded at the town hall". After his death the property passed to the hands of his son of the same name who sold it in 1443.
From his new seat at Kostelec, Kuneš of Dubá, called Rozkoš, organised raids in the surrounding area while battling on the side of the Catholic opponents of George of Poděbrady. Since Kuneš endorsed the so-called "peace of the land" charter, however, and in spite of this he ravaged the Czech Kingdom "with fire, weapons and plunder", Kostelec was besieged in 1449, albeit to no end. After a short truce the castle was besieged for the second time by the armies of George of Poděbrady and Prague City in May 1450. Despite the fact that the assisting emergency forces of one of the allies, Saxon Duke Friedrich, broke away in the direction of the town of Most, thus relieving the besieged castle and confining part of the Czech army, Kostelec was nevertheless soon conquered. The military and political importance of the castle is reflected in the fact that George of Poděbrady himself participated in both sieges. Jan Sádlo the Younger was also reported to have been present in the invaders' camp, evidently for his knowledge of the castle and its weaknesses.
The Kostelec estate was divided up and the deserted castle was later possessed by Zdeněk of Sternberg who fortified and enlarged it (interesting to note is the fact that Kuneš of Dubá received 1,550 kopa ["three-score"] of Czech groschen in compensation for the confiscated castle). After his rift with the king when he placed himself at the head of the so-called Fraternity of Zelená Hora, the king's army lay siege in 1467 to six of his castles, among them Kostelec which was conquered, destroyed and never restored...

The abandoned castle was later incorporated into the Konopiště estate and left to its own fate. At the end of the 18th century it belonged to the counts of Vrtba, the owners of neighbouring Týnec. The castle grounds were evidently partly cleared, the route across the moat was made accessible with stone steps and some of the wall crowns were secured. A summer-house looking out onto the Sázava valley was probably constructed in the upper courtyard, however, nothing remains of it today.
During the last century the romantic castle ruins frequently attracted various romantics, artists and those interested in Czech history. One of these visitors was the czech poet Karel Hynek Mácha who travelled here and sketched the castle in his album during his journeys through the Sázava region.
The civic association Order of the Knights of the White Cross is currently trying to save the castle. The aim of the Order is statically to secure the remains of the castle buildings and walls and to create the space and surroundings for the presentation of the castle and its history along with cultural events, set in a historical environment...
