I suspect your area is big enough without coming up to Angouleme - but it does rather depend on what you like to do. If you want to see around 30 Bugattis in proper use, plus a vast array of other classics, and a lovely atmosphere, then perhaps hit Angouleme. http://www.circuitdesremparts.com/en/

But in terms of driving roads then it's probably not worth the detour.... The chatueaux are also spread more thinly than, say, the Dordogne.

I forgot - there is one chateau in the Dordogne with amazing gardens and worth a look (the chateau is really only a big house) - Château de Marqueyssac - have a google and you will see what I mean. Not as impressive as, say Chenonceau though (which I assume you hit on your Loire trip)

Nick