Several things have an impact on belt life. Time certainly is one, which is especially critical if you do not drive much. A belt that sits starts to stiffen into a position. So I would rather have a 2 year old 20k miles belt over a ten year old belt with only 2k miles.
Visual inspection (unless there is something obvious) is a crap shoot (IMHO). That 10 year old belt with only 2k will look good.... and then you get to rebuild an engine.
Think of it like tires.... would you run a 20 year old tire with only 2k miles?