Scottish Borders: See
Abbeys: these sprang up in Norman times and flourished in the Middle Ages, but were left to ruin after the Reformation - the 16th-century break with Roman Catholicism. Find them at Jedburgh, Melrose, Kelso and Dryburgh near St Boswells.
Stately Homes: built for lordly living, not for defence. Those which can be visited include Floors in Kelso, Mellerstain and Thirlestane near Melrose, and Paxton near Berwick.
Castles in the Borders are mostly Peel Towers and turrets. They were residences and farms fortified against bandits, rather than military garrisons. Most are scrappy ruins, some have been incorporated into later buildings. One you can visit is Smailholm Tower between Kelso and Melrose.
St Abb's Head is the most scenic section of the Berwickshire cliffs, with sea birds whirling over a tumult of waters. Take the minor road east from Coldingham, park at the head of St Abb's village (don't take a car down the steep narrow lane to the harbour) and walk across the fields to the Head.
See Berwick-upon-Tweed for places right on the border: Norham Castle just south of it in Northumberland, and Paxton House just west in Berwickshire.
Scotland's Gardens opens up private gardens once a year in summer, with all proceeds going to charity. There are about 25 participating gardens in this region, dates staggered so there's one open most weekends.