I agree totally.
I always thought of Ray Ban as "the name" for sunglasses, not designer, like Prada, D&G, Bulgari etc. Which is why I wouldn't object to buying Ray Ban, but I'd never look at those other names.
It is the same for most things I buy, I'll buy a brand that has a reputation for building good quality in that type of product (Toshiba, Panasonic, Sony for television, for example), as opposed to Matsui or Silverline as "unbranded" products, or B&O and their ilk at the "designer" end of the market.
As to wearing sunglasses or not, if they are proper sunglasses, they will protect your eyes from harmful rays. I prefer not to wear them as well, but when it is really bright or there are no clouds in the sky, I do wear them.