The castle was founded as a fort in the 13th century by the Hrabišic family, which resided at the Osek Castle. Not earlier than 1527, the Lobkowicz family replaced the fort with a one-wing Renaissance palace. Marie Polyxena of Talmberk and Lobkowicz, the daughter of the František Josef of Lobkowicz, the last member of the family, married Maxmillian, Count of Waldstein in 1642. Their son Jan Bedřich, Count of Waldstein, later the Archbishop of Prague, was apprised with the French architect and painter Jean Baptiste Mathey, and brought him to Duchcov for the purpose of rebuilding the palace.