This timber castle (later reinforced with stone) changed hands repeatedly as the Normans warred with the native Welsh, who were firmly ensconced in this area of Mid Wales. The castle first fell to Owain Gwynedd in 1136. The castle's ownership switched at least three more times before being captured by Llywelyn the Great in 1221. Scholars believe that Llywelyn probably razed the castle and rebuilt another one in its place. It is not mentioned again until Edward I erected what is now known as Aberystwyth Castle a mile north of this site.
Aberystwyth Castle was built, along with Flint Castle, Rhuddlan Castle and Builth Wells castle, by King Edward I as part of his campaign against the Welsh. Aberystwyth Castle, started in 1277, was a particularly taxing job. It still was not completed by 1282 when the Welsh briefly captured and burned it. Construction was finally completed in 1289 at great expense to the crown. The castle was subjected to a lengthy siege during the revolt of Madog ap Llywelyn in 1294-5.
By 1307 the castle was thriving enough that people had made their homes next to its walls, and the town was decreed Llanbadarn Gaerog (Fortified Llanbadarn). But the town was commonly referred to by the castle's name, as is the case today.
The castle changed hands several times as the Welsh and English warred throughout Wales, including by Owain Glyndwr, who took possession of the castle in 1404. The English soon after recaptured the castle, but by 1408, peace having been struck, the castle began to fall into disrepair. Still, in 1637 Aberystwyth Castle was designated as a royal mint by Charles I. The mint struck silver coins. This association became the castle's downfall, as the mint's operator, who had been made wealthy by his job, raised a regiment of Royalist soldiers during the English Civil War. This made the castle a target of Oliver Cromwell, who consigned the castle to slighting in 1649.