If your scared, your not riding enough.
getting old isn't a problem for muscles its bones and joints that wear out.
if your not getting knee, ankle, spine, wrist or shoulder problems then your doing well.
for me i just don't get out on the bike enough i am lucky if i get out once a week during winter.
i loose confidence in my skills.