I think from your description that the back box (behind the rear wheel?) is just an acoustic trim and you are retaining the silencer which sits under the front wing by the drivers door. If so, removal of it will not make a huge difference to the sound.
