The leader of the party is the one that makes decisions and respond to flags as they normally do. The rest of the party follow the lead as if it had a "defend" flag on it.
I only use parties if I want to defend a weak hero like a wizard (Fighter, Cleric, Wizard, and Elf makes a good group). For regular city defense it's better to not have parties so all the individual heroes can respond to threats as well as manage their own needs (shopping, taxes, etc.).
Once I'm ready to start setting big attack flags for the "boss" or end encounter I often make a group for each elf with a fighter in lead. This way the elves don't run off ahead and get into the fight too soon.