Have you checked how true your rear wheel is running? I imagine that if it was slightly out, the belt would tend to run out from the centre of the rear pulley.
I've seen some of the pre production cars at the factory with a pronounced runout.
Jays Former Morgan owner. Gone but hopefully not forgotten!
