Samtskhe-Javakheti: See
Besides the famous Vardzia (a 12th-century cave monastery and city carved out of a cliff overlooking a river gorge), there are Upper Vardzia, Khertvisi Castle and Vanis Kvabebi (another cave monastery) closeby. See all the information at the Vardzia article.
Furthermore, near Akhaltsikhe do not miss the interesting Sapara Monastery.