Bronllys Castle is thought to occupy a pre-Norman princely court or llys, sited above the floodplain of Afon Llynfi. In the late 11th or early 12th century Richard fitz Pons founded an earthwork motte and bailey fortress on the site.
In the mid 12th century Walter de Clifford founded the stone castle, when crowning the ditched motte with the 'principal tower', which was damaged by fire in 1165. In the 13th century a two storey round keep with a vaulted basement incorporated this tower.
A forebuilding once protected the first floor entrance and the keep was crowned with a three windowed storey, in the 14th century. To the north a now landscaped pentagonal enclosure of two wards, is defined by sections of banks and ditches.