At the time of the Norman Conquest, Danby was part of the Saxon estate of Crumberclive. The estate was given to the Fitz Baldric Family who may have built the first castle in the area, at Castleton west of Danby. The Fitz Baldrics lost their land when they rebelled against the king and it passed on to the Brus or Bruce family who rebuilt the earth and timber castle in stone. Robert de Brus founded Guisborough Priory in 1119. In 1272 the Brus family died childless, and the Estate passed to the Thwengs, and then to the Latimers, whose Arms (along with the Roos family) appear on the North Front of Danby Castle.
With the death of the last of the Latimers, at the end of the 14th century, the estate passed to the famous Neville family. They remodelled the SouthRange, where their Arms can be seen on the south wall of the Courtroom. In the 16th century John Neville, Lord Latimer of Danby, who owned Danby Estate and DanbyCastle, married Catherine Parr, (Henry VIII's last wife,) and their marital home was DanbyCastle.
DanbyCastle, now in part a ruin, but also the home of the Danby Court Leet, is reputed to be one of the earliest examples of a fortified but principally domestic castle. The many ensuite guardrobes are ample evidence of the 'modern' features of this castle which sits in a strategic position overlooking the River Esk valley.
Later in the same century the Estate passed to the Danvers family and Sir Henry Danvers was created Earl of Danby. In the 17th century Danby was bought by John Dawnay, later created Viscount Downe. The Estate has remained with the Dawnay family ever since. The SouthRange of the Castle was converted into a manorial Court-house with the former private chamber (solar) being divided into a Court Room, where the Court Leet still meets, and a Jury Room. As regards the rest of the old mansion, the South-East tower became and still remains, a farm house. Farm buildings were added and some original sections have fallen into ruins. Repairs were made in the 19th century and the SouthRange was restored in c. 1960.
Thank you to Dawnay Estates for providing the above information - www.dawnay.co.uk