I do use sidescreens for long motorway journeys to cut down wind noise and buffeting, but I've never really considered the possibility of a flying stone hitting me when driving with the sidescreens off - something which I do quite a lot.
But in 20 years of driving Morgans I've never had an object impact the sidescreens, so I think it's a pretty remote risk. As others have said, you can let worrying about possibilities spoil your enjoyment of driving.
