If you're just starting then your muscles and joints will be getting used to the extra strains, although joint pain probably isn't good. If you're concerned then see a doctor or at least take a few rest days and see if that helps.
Also, try warming up and stretching before you ride.