Steve,
I've done this and it goes back without any problems. From memory its leather covered plastic.
I do seem to recall taking out the trim strip to tighten up the mirrors and then realising that I needn't have. Might be worth checking if you can get a ring spanner on the mirror nut before taking it apart.
