No not just you.....and I had the same feelings on motorcycles....though it was easier when driving abroad because you could either just lift your finger on your left hand to wave or alternatively take your left leg off the footpeg and straighten your leg. With oncoming vehicles being on your left it was easy, you cannot do the same finger wave in UK because it interferes with you throttle control.
Even then though there were people on certain marques that seemed to think that waving or acknowledging a fellow enthusiast was somehow below them. The less said about them the better!
Sometimes my mind is elsewhere when I am riding/driving or, like you, I am simply too busy concentrating on what I am doing, and I miss an oncoming wave....I feel guilty....but the guilt passes.....and seems to spur me on to ensure that I do not miss the next one.
So wave if you can....and don't worry if you can't.....it is not a crime!