I'm only a nice empire in half of my playthroughs. But there is a mystical dark influence that affects players who play for long enough and causes them to take pride in being xenophobe. So far I have resisted the corruption to cross fully to the other side.
For me, non-Egal / non-Phile allows enough control that I don't need to be Phobe.
It is restrictive but there's still a lot of distinct combos in the Auth / Mat / Spi / Mil / Pac space -- and even Phobe has some fun potential with Pompous Purists (which allows you to be Phobe but not use the "Xenophobe Playstyle").