"Should've" and "should of" sound very similar in my accent, its just one of those things that gets mistaken once and perpetuated. To my mind, a lot of slang and bad wording of things comes when you're working hard under difficult conditions. Its rare that academics speak badly, but (for instance) engineers and mechanics with a similar level of education will swear, curse and generally abuse the language to get their point across.