Reply, yes. Answer the question, errm no I don't think so.
"Excuse me, can you direct me to the bus station?"
"If you have to ask you'll never know".
"Excuse me, do you think my bum looks big in this?"
"If you have to ask you'll never know".
At best it's a statement which implies and answer, but an answer is it actually exactly 100% what it isn't. A question requests information, and answer provides it. He's none the wiser, therefore it's not really an answer to the question. It's about as useful as:
"Fancy a pint?"
"I feel really pretty today."
Thus, and in conclusion, bollocks.
The explanation you provided afterwards, that's a partial answer. It doesn't offer any enlightenment, but it does tell the reader what they need to do to find it. (In this case, go and try it).