Scott, I'm with Lorne here. His two suggestions are, after so many attempts at trying to find a solution myself, the best. I also found that the roof up sides off option is the most comfortable. No peaked sweaty hat required and no burning from direct sunlight.
When growing up... a long time ago now! our family 4-4 had a zip out rear window and that was the best combo ever, roof up, sides out and rear window out. It achieved two things, greatly reduced the wind buffeting both noise and physical thumping and best of all good air through flow.
The rear wind block positioned behind the seats is decorating my house as frankly I really cannot see any change to buffeting or noise that it made... I'd love to see a wind tunnel test and if micro positioning helps.
My current car, a millennium plus4, doesn't go on great adventures anymore but I do intend to get Steve Summers to install a zip out window in the near future. I have and don't suggest it as a solution but if on country roads I pop the rear roof hold downs and this helps air flow. I certainly wouldn't recommend this for motorway or fast driving. I haven't tried it at speed but have had visions of levitating roofs and worse so buyer of my practices beware!
One other advantage of roof up is town driving.... traffic lights and congestion. People are far less likely to engage with you and this can be very distracting when you should be concentrating on driving!