What is white? Shine white light through a prism and all colors are separated. White is all the colors integrated in one. It is all and one. Saruman in his vanity and pride seeks to dominate. Therefore he becomes Saruman of many colors. In his pride he has given up status and clarity. He is now, like the other Istari, subject to white. Saruman the white is reborn in Gandalf the White. And he, with no trouble whatsoever, strips Saruman of his staff and position.
It is a symbol, yes, but an interesting way to see it none the less.