this is not a shame. a good vee brake can perform as well as a magura, and with dedicated vee mounts they're usually lighter too.
i'd invest in some better brake levers these are my favourites. i like the split clamp design and they don't end up all wobly and loose like avid levers do. best price on a pair, they usually throw in new brake cables which will also boost your braking performance. so will some heatsink vee pads.
obviously a better freewheel would improve things no end as would a better bar/stem combo.