Actually the swastika is the world's oldest complex symbol, over 8500 years old, and has been used world-wide by countless civilizations often as a symbol of good fortune. Especially Asian cultures also draw the swastika as a sauvastika (left-facing), with the same meaning, but in a few cultures the sauvastika is evil while the swastika is good. It remains very widely used in Asia, and in some Western countries depending on context, but this has considerably decreased thanks to most ignorant people who believe it's exclusively a Nazi symbol.
But yeah I agree that perhaps at least the Slavic symbols are inappropriate in the flags.
I wasn't trying to say that the swastika was only used by Nazis but that Nazis only used the swastika. They even wore it as a badge on their arms. So why should they keep the rest of the flag and just change the focal point?
Be that as it may, there won't be a swastika on the flags anyways... Nevertheless my logic applies to any other german symbol which is placed in the middle of the flag.