The wooden spacers make a huge difference. I've experimented with different thicknesses, and the compromise for what you need is a shallow runner of the bolt head thickness, so about a quarter inch. This will be the minimum to stop runner distortion and give maximum clearance. If you don't want to mess about with wood, try spacer washers under each bolt, but it will be more fiddly to fit.
