Hi Will,
I am rebuilding a 1985 4/4 and have removed the rear panel.The panel is held by wood screws at the top where it is folded over but the screws are covered by the interior covering ( in my case carpet material) The "folding over" of the panel at the top will need to be slightly un folded which may crack the paint.(it cracked the paint and filler underneath on mine). I realise my car is an earlier and different model but from your description it sounds as if it is constructed in a similar way.
I can PM you some photos if it will help explain the above
Philb