The reason our brains allows us to 'see' colour is an evolutionary benefit, it allows us to distinguish different frequencies of light. It's said that bat's can 'hear' colours, they have a different colour assigned to what type of texture an object is because that object will reflect their sonar differently. Both us as bats receive different frequencies from the world around us and assign a colour we just do it in different ways.
So, if all that a colour is is a label that animals' brain will assign to a frequency, what's to stop, say, your friend assigning orange to what your brain has assigned to blue. Lets say the amber light on tragic lights. Imagine if your friends brain has assigned the frequency that comes from that amber light as blue, but blue to him would be called orange as that's what he's lived with for his entire life.
It would be completely untestable too as it's impossible to describe a colour (how could you explain red to a person who's been blind from birth?)
Could it be possible other users of TF see it like this?