Let's see, I'm 46, fairly-fit (certified as personal trainer), intelligent (720 GMAT), youthful (look younger, dress well, tech literate), and vibrant, and I do fear aging. My biggest concern is the loss of intellect that accompanies age. I haven't had cosmetic surgery, but I am not averse to it.
Caring about one's appearance is not ageist, and I imagine it is more often the anxiety of someone that is youthful and attractive.
Personally, ageist comments have come from 20-somethings, not older people or managers. According to a study, management typically has a higher regard for older workers, in terms of productivity, as well as with people skills, although managers also think younger workers can be more analytical.
On an intellectual level, older people tend to be neurologically slower, although the top few percent will increase their IQ up until the early 40's. After that, there is a typical, slowing and loss of intellect.