Without sidescreens my glasses tend to leave my nose without warning above 80 km/h. So my rule of thumb is: short, scenic and winding drives where I rarely reach 80 km/h: sidescreens removed, all other occasions: sidescreens up.
For my feeling aesthetics are by far better without sidescreens.
