English can form adjectives from the past participles of verbs. Consider:
- break:broken the vase was broken -> the broken vase
- fall:fallen his popularity has fallen recently -> his fallen popularity
- bake:baked the bread was baked in a brick oven -> baked bread
- age:aged the wine is aged in oak barrels -> aged wine
Concerning age, the combination adjective teenage almost never appears in current usage with the past participle final -d. You almost always read or hear
her teenage daughter
not
her teenaged daughter
English dictionaries do include the word teenaged as an adjective, along with teenage, although teenaged (1952) appears several decades later than teenage (1921) according to the Online Etymological Dictionary. Teenage is more in line with new age (new age music) and ice age (ice age relics) with age as a noun, not a verb, i.e., a period of time. Similarly, there is mid-life crisis, not mid-lived crisis, again related to a period of time.
What about the adjective middle-aged? First, it almost always appears with a hyphen in written English (unlike teenage/teenaged). And the situation is nearly the opposite of the case for teenage/teenaged. You almost always see the forms middle-aged men, middle-aged women, or simply the middle-aged. Kind of interesting, too, that we have teenagers but not middle-agers. So, my linguistic intuitions were a bit jarred to read in print “middle-age men in shorts” recently — without the final participial -d. Is that because middle-aged is more verbal, referencing a weathering process, rather than a time period? But then why middle-age spread (abdominal fat accumulated in mid-life) and not middle-aged spread?
What about other cases of modifiers that usually take the final -d? Are the d-less alternates as acceptable?
- three-fingered glove/three-finger glove
- left-handed presidents/left-hand presidents
- right-angled turn/right-angle turn
- bare-fisted fight/bare-fist fight
- three-toed sloth/three-toe sloth
- three-cornered hat/three-corner hat
- heavy-handed methods/heavy-hand methods
- strong-armed tactics/strong-arm tactics
- underhand pitch/underhanded pitch — these mean something quite different 🙂
Word Travels 
I think it’s because if you say ‘new-age men’ you mean men whose outlook is characteristic of New Age philosophy, rather than their being newly-aged like newly-aged cheese, where the ‘-ed’ makes it a past participle so it accepts an adverbial modifier. By the same token middle-age men would refer to habitants of the Middle Ages or of any period falling between two others, say between the Age of Enlightenment and the Age of Romance (an enlightened romantic 😉 ). Or to those who reflect attributes thought to characterize such periods…
And in fact we often use -age in precisely that kind of allusive way. We speak of ‘teenage music’ not to describe music composed 13 to 19 years ago, but rather music that appeals to adolescent tastes. ‘Middle-age spread’ references the fact of the expanded waistlines of many people in their middle years; it doesn’t mean the adipose tissue itself is 35 to 55 years old. So we could construct a formalism in which we use aged to mean how old (‘…died yesterday at Brighton, aged 91’) and age as a pointer to some remote object. Well, we could, but I doubt anybody would start talking about moderately-aged men in shorts…