Without listening to the noise it is difficult to diagnose. Richard is probably correct if that is indeed the source.
Just to throw in a curved ball, is it possible that does not come from inside the diff at all, and is could be due to other issues .The fact that the noise is different power on/power off makes me wonder if the diff (or some other part of the rear axle) is hitting the transmission tunnel (or the top "shelf"?) under power. This will result in a very metallic whirr which will probably be speed dependent. However this theory does not fit the fact that the noise is prevalent on a light throttle.
Have you checked that the exhaust system does not contact the chassis , or bodywork at any point ? Clutching at straws here I admit .
The 4/4 Sport is a wonderful car, in many ways truer to historic Morgan DNA than any post 2000 model, and I am sure this can be sorted.
