Its all in the mind. Your perception of risk is just that. My perception is probably quite different.
The only time I have felt a little uneasy in the way you describe was while driving through a very narrow gorge in the Pyrenees - overhung with vertical rock, and noticing fallen stones and little patches of asphalt where deep dents had been infilled.
Directed me NOT to use the horns in case the sound disturbed the rock face above.
Drive on regardless.
