personally i would regrind, re setup, make sure you have a good bleed and get a 4 finger lever, with a good bleed you will hardly notice the sponginess, i wouldn't take the booster off till you've tried that yet. boosters do increase your hold, and with a 4 finger lever it should work amazingly, if it doesn't after all that then take the booster off, do a process off elimination
out of interest what year brake and pads are you using?