I'm with you there. I'm pretty sure the tolerances for headset/fork/stem set ups don't have to be too tight!
Another alternative is you could shave off 1mm off the top of your headtube,
or 1mm off the bottom of your stem.
or 0.5mm of the top and bottom of your headtube,
or 0.25 off the bottom headset cup, 0.25 off the bottom of the steerer, 0.25mm off the top of your headset cup and 0.25 off the top of your headtube!
Saying that though you'd effect your riding position by 1mm and that could be a total nightmare!!!!!!