When driving along, under load (IE, accelerating, or cruising at a constant speed with the power on, ie - not coasting) there is no noise from the back of the car - just how it should be

BUT
As soon as I let off the throttle, at any speed, there is a rubbing/knocking noise coming from the diff/driveshafts. It sounds like something is chaifing.
Initially, I thought it was the prop on the rear beam as it is very close, but after fitting a new uprated diff mount bush, and lowering the diff by 3mm, the noise is still there.
Its not a loud knocking or banging, its a dull scratchy rubbing noise, and is definately coming from a driveshaft or the diff output flanges (I know this due to the speed of the road wheel in proximity to the noise, the prop is spinning too fast for the noise)
The noise starts from about 5mph all the way up thru the speed range.
I have changed a driveshafts on both sides as I thought it was that, but no joy, and upon listening to the noise while coasting at about 20mph, it sounds like it is coming from the middle of the rear beam, indicating the diff.

The driveshaft angles are not excessive at all, they are approximately 10 degrees pointing down from the diff, which is a safe working angle.
Anyone with any ideas asto the problem, I would be grateful

Cheers
Paul