Not in the slightest mate! I've seen people 40+ getting into this sport, to be honest it's one thing that gave me the confidence to start. Even though I'm only 18 my fitness is pretty poor due to having an accident last year and being lazy since!
Age is nothing but a number and a state of mind, get out and get riding!