I had this problem too, and solved it by slacking off the nut and pulling out the mirror and stem. I removed the mirror from the stem; secured the stem in a bench vice and gave it a little light persuading. It's been perfect ever since.
As dmh has implied, you'd have thought that after 75 years of building the 4 wheeler in various guises that MMC would have cracked the whole mirror thing, (but not in a 7 year's bad luck sort of way).
