Home Trending Latest threads New posts Developer diaries
Jump to latest Follow Reply
  • We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
Stellaris 54225 Bug Reports 37615 Console edition 1312 Suggestions 21772 Tech Support 3216 Multiplayer 392 User Mods 4795 Stellaris AAR (After Action Reports)
Previous dev diary Next dev diary
Show only dev responses

Stellaris Dev Diary #172 - Reworking the AI

Bonjour everyone, it’s the French Paradox speaking! For those who don’t know me, I’ve joined the Stellaris team this December after a year and a half as a programmer on Europa Universalis IV.

Today, we are gonna talk about AI.

pasted image 0.png

A good introduction for those new to the field​

Fifty Shades of AI
There are several AI modules in Stellaris. For historical reasons we call them “ministers” as each one is supposed to handle a specific role in an AI empire.

There are 3 broad kinds:
  • The AI foreign minister handles diplomacy, federations, galactic community, peace deals and the like
  • The AI interior minister is in charge of the economy. He keeps budgets and order constructions, both civil and military.
  • The AI military minister is in command of all troops and military fleets, and also responsible for laying out strategic plans when at war.
For each of those ministries there are different “ministers” there are several options that can be selected for every empire in the game. All of those have generic one which behaves more or less like we’d expect a player to and is used for most AI empires. Then we have a bunch of specialized ones for special tags such as space monsters, fallen empires, crisis, marauders and the like.

As almost everything in our games, AI is configurable in script for our modders, although I’m not exactly sure what would happen if you assigned a space monster military AI to the caravaneers ;)

In guise of a welcoming gift when I joined the team, I was tasked with reworking the military one...

The Military AI
To give you a little bit of background, there were several generations of military AIs in Stellaris. The generic one (used by most “classic” empires) was redone by the great @sidestep last year, while the more specialized ones (crisis, space monsters) have kept close to what they were on release. In the midst of the sad and dark swedish winter, I managed to bring some improvements that I’ll showcase today.

First of all, I worked on visualization to help us debug how the AI “thinks”. Funny thing is, it already made it look “better” to audiences even if it didn’t actually change any behaviour. It’s actually something that’s been observed in video games: a good AI tells you what it does, which makes it look smarter. One of my favourite examples of that would be the enemies in FEAR.

So by typing 'debug_ai' in the console and observing an AI empire, you can see what it has in mind:

pasted image 0 (1).png

“I don't even see the code. All I see is blonde, brunette, redhead. Hey uh, you want a drink?”​

As a simple analogy, imagine that the AI has a war minister that looks at the big picture and rates every potential target, a general staff who assign fleets to some of those objectives, and then admirals who try to lead those fleets on a tactical level to achieve those objectives.

The skulls on top of each system shows military objectives that the AI is considering (the war minister). Red ones are the ones they selected and committed some fleets to, while green ones are other options they haven’t retained for now. Finally for each individual fleet, in those task forces, you see what they are doing at present.

In our screenshot example, the AI decided that taking Tiralam was the most important objective with a score of 4500, and that they estimated that at least 11.2k fleet power was needed to accomplish this. They committed the Kilik Armada, the Jinki-Ki-Ti Armada and the the Grekki Armada to this. Since it makes little tactical sense to attack in a dispersed formation, the AI issued orders to regroup in Broon’s Singularity before proceeding on the attack (something we improved in this patch).

For convenience, the summary is also visible in the outliner:

pasted image 0 (2).png

As seen from the other side of that war​

That change alone allowed us to see where the AI was a bit weak and also made evident a few bugs in the production AI that we promptly fixed. A funny one was that in some cases a fleet would end up assigned to two different fleet groups, nicely simulating two admirals fighting over command of a fleet and issuing contradictory orders every day.

Crisis AI
The next step was to rewrite the various crisis to use the generic AI, so that any effort spent on making better would benefit all. In patch 2.6 the specific AI of the Khan, the Prethoryn, the Unbidden and the Contingency will use the same AI as the “standard” empires, with a few twists to still retain their personality.

Without spoiling every secret, here’s a few ideas:
  • The Khan doesn’t really believe in defense and will try to beat the closest systems into submission
  • The Prethoryn will swarm in every direction they can
  • The Contingency will systematically try to stop the biggest threat to the galaxy, until nothing remains
  • The Unbidden will be harder to predict, but there’s reason behind their alien way of acting.
One of the biggest challenges we faced was assigning fleets to objectives. Matching X fleets with Y out of Z objectives is not an easy task. Do we try to accomplish as many objectives as we can at the risk of spreading too thin or accomplishing nothing of value? Should we instead focus on the most valuable target and possibly end up in a big fight that we could have avoided? How often should we reconsider our options?

The current version solves this by putting a fleet power value on every target, then grabbing fleets by order of priority until it either has enough to accomplish the objective, or go over the next one. This approach showed its limits when we plugged the crisis AI into it, as it relies a lot on the size of available fleets (it doesn’t know how to split them, it can only merge them).

Teaching the AI how to split fleets proved quite interesting:

pasted image 0 (3).png

What shall we do with this knowledge?​

It took several tries to find a good balance, as the AI tended to split too much (most objectives don’t call for that much fleet power, unless you’re fighting your enemy main fleet). In the end, after trying some complex strategies such as keeping statistics on accomplished objectives and deriving a good target number from that, a simpler approach turned out more efficient: put all the nation’s offensive fleet power into one stack, and then consider splitting in 2,3 or more depending on how confident the AI feels about its military power versus its foes.

Knowing some of you like to mod our AI, here’s some new defines you may want to play with once all that hits the shelves.

Code: 
# Objective values
HORDE_INVASION_PLANNING_DEPTH = 5    # How far out does the Horde AI looks for invasion targets (in system hops)
SWARM_INVASION_PLANNING_DEPTH = 5    # How far out does the Swarm AI looks for invasion targets (in system hops)
SWARM_POP_TARGET_MULT = 1.0            # Extra target scoring for swarm (multiplied by number of edible pop on the planet)
CONTINGENCY_MEGASTRUCTURE_EXTRA_VALUE = 4    # How attractive are megastructures to the Contingency (added to the base value of 1)
UNBIDDEN_PORTAL_EXTRA_VALUE = 20            # How much does the Unbidden want to defend their portal (compared to base value of 1)
UNBIDDEN_BYPASSES_EXTRA_VALUE = 4            # How attractive are bypasses to the Unbidden (added to the base value of 1)
UNBIDDEN_RIVALS_EXTRA_VALUE = 10            # Extra target scoring for rival invaders (Aberrant and Vehement)
UNBIDDEN_TARGET_EXTRA_VALUE = 10            # Extra target scoring for randomly chosen nemesis
UNBIDDEN_PSIONIC_CONQUER_DESIRE = 20        # Extra weight added to psionic empires when rolling a nemesis (base 1 + number of owned bypasses)
UNBIDDEN_CHOSEN_ONE_CONQUER_DESIRE = 50        # Extra weight added to empire lead by the chosen one when rolling a nemesis (base 1 + number of owned bypasses)

# Fleet sizing
OFFENSE_VS_DEFENSE_STRATEGY_ALLOTMENT = 0.75 # How much of its fleet power should a country with 1.0 aggressiveness should try to commit to offensive missions
AVERAGE_FLEET_SIZE_FACTOR    = 0.05            # Ballpark estimate of the minimum size a fleet should be in relation to total fleet power
OWN_FLEET_POWER_FACTOR = 1.0                # How much does AI count its own fleet power when evaluating forces
ALLY_FLEET_POWER_FACTOR = 0.5                # How much does AI count ally fleet power when evaluating forces
ENEMY_FLEET_POWER_FACTOR = 1.0                # How much does AI count enemy fleet power when evaluating forces
FLEET_SUPERIORITY_FACTOR = 1.5                # How stronger should the AI be before it starts considering splitting fleets (fleet count = relative strength / this factor)
CRISIS_FLEET_SUPERIORITY_FACTOR = 1.0        # Same as previous but will be compared to the strongest foe in the universe

Most of those changes will be delivered in the patch coming alongside the Federations release (2.6.0), but not all. As you may imagine, changes to the military AI are quite impactful and we don’t want to release the changes without enough testing, so some of them will be delivered in the first support patch (2.6.1).

And with that, I shall leave you with @sidestep one last time.
 
Last edited by a moderator:
  • 2Like
Reactions:
Sort by date Sort by reaction score
semisweet said:
I'm wondering how the AI responds once it reaches its goals? Seems like this is an important consideration since it obviously wouldn't be good for empires to stagnate just because they reached some number.
Click to expand...
Been answered by a dev. They never cease to do anything and if there happens to not be a goal or focus goal, the AI reverts to the weighting system (which I assume is improved as well).
 
semisweet said:
I know not everyone agrees on this, but I'd rather have small details like this that aren't balanced than not have them at all. Not every empire is destined to own the universe. Sometimes some fall to cruel fates. I like that. I get that it's more important in multiplayer where people aren't there to roleplay a tragic fate, but in single player it adds a lot of flavor to the universe.
Click to expand...

It was more of an additional concern than an argument against the inclusion. If anything it was perhaps a subtle nudge to give psionic empires a bit of a buff.
 
Consider some logic where the AI builds a main fleet and then spread the rest of the assets into other fleets. The % split of total available fleet capacity is something you need to toy around with :)

Nice to see some updates to the AI, and thank you very much for the insights into the workings of the game!
 
Would you be willing to, at some point @sidestep , be willing to look at how the AI picks traditions and allow it to pick traditions that are not typically just the original sets?
 
Orion88 said:
Also in relation to that, one thing I have always hoped the military AI´s could do in these space 4X strategy games is build fleets that have different kinds of ship designs compared to the other AIs (So that one AI empire could use more beam based weapons for example and another more missiles etc.) So that the various empires would feel different in combat as well.
Click to expand...
It already does this. Extensively, even. Each AI "Personality" (such as "Erudite Explorers" or "Trade League" or "Hegemonic Imperialist") as a favored mix of weapons and a favored mix of defenses. This is one of the stupidest things about the Stellaris AI, IMO; because they mostly build their ships with an imbalance, it's easy for the player to exploit it... and you can predict the imbalance in advance just by looking at the line under the empire name in the Diplomacy screen!
Orion88 said:
human players can adjust their strategies and tactics in relation to what the AI does, but AIs usually don´t seem to have this ability, and instead they just do the same mistakes over and over again. (And once you learn what those mistakes are it becomes easy to exploit them.) So any chance of giving the AI at least some ability to adjust to what ever the human player is doing against it?
Click to expand...
Wait, which way do you want it?!? If the AI just switches to doing whatever best counters the player (or other AI that it's currently fighting, I suppose), then the AIs will obviously just end up doing the same thing that the player is doing - balanced ships in the early game, then going heavy on the deeply overpowered Neutron Launchers later on, never using missiles (except maybe torp 'vettes in the late game) or strike craft, etc. If they instead keep doing their current thing - even if you were to randomize the strategies so that it was actually necessary to scout their ships before countering them - you could (and should) continue to exploit the heck out of it as they "make the same mistakes again and again".

Also, the AI does, in fact, already refit its ships somewhat in response to enemy ships. For example, if you're enough of a fool to put missile batteries on your starbases, the AI will show up packing enough point defense to neutralize every single missile you launch. You usually can't convince the AI to diverge too far from its favored weapon mix - if you start running 100% shields against an AI that you recently saw running 100% lasers, it might bend enough to go 40% kinetics 60% lasers instead - but it does adapt to some degree.

Well, except the various spaceborne aliens and crises (including the Fallen and Awakened Empires). Those never refit their ships, so you can (and should) counter-fit them hard.
 
UltimateTobi said:
Been answered by a dev. They never cease to do anything and if there happens to not be a goal or focus goal, the AI reverts to the weighting system (which I assume is improved as well).
Click to expand...

It wasn't really answered as far as I remember. That post was about population growth. It cannot be assumed that the same applies to regular resources. If it does my other question still stands, which is, how does it handle it? The way the AI was described was that before you reach the goal the priority is proportional to how much of a difference between your current income and goal income there is. Once you pass the goals that difference becomes negative. What does the AI do then? Does it switch to a new analysis method that deals with ratios rather than differences?
 
The AI switches to a new plan if the last plan was reached, and if there are no plans, it builds buildings based on the building weights system that exists currently (i.e. before 2.6).
 
Shadowstrike said:
The AI switches to a new plan if the last plan was reached, and if there are no plans, it builds buildings based on the building weights system that exists currently (i.e. before 2.6).
Click to expand...

I see. I must have missed that post when looking through. Thanks for explaining. Interesting that the weights will still be used. I thought a big part of the idea was eliminating them completely to simplify modding and updates.
 
semisweet said:
I see. I must have missed that post when looking through. Thanks for explaining. Interesting that the weights will still be used. I thought a big part of the idea was eliminating them completely to simplify modding and updates.
Click to expand...
They're a fallback. I imagine they wont actually get used much, so its fine if they're not as carefully updated.
 
Well.. I guess, this whole The Plan TM thingie also applies for the player's sector-AI and for automated building the player's planets, if set to,.. let's say "Science World"? Last time I checked, the AI was only building much of City districts on a science world... just saying. :D
 
Orion88 said:
And finally, do you have all the necessary processing power there at Paradox to test and make the AI as smart as possible? Since if no perhaps we members of the community could give you some more processing power to run small tests on our systems to make the AI as smart as possible. @grekulf (I think there was some news recently about SETI home project closing soon, so if we can´t use our home computers to find intelligent life from the universe, perhaps we can create some inside Stellaris instead. ;) )
Click to expand...

Also one thing to add to this is: Could there be perhaps some AI vs. human player telemetry gathered from the community that could then be send to Paradox for analysis, so that the developers could recognize more easily where the AI is particularly weak against human players?


SeekingEtermity said:
It already does this. Extensively, even. Each AI "Personality" (such as "Erudite Explorers" or "Trade League" or "Hegemonic Imperialist") as a favored mix of weapons and a favored mix of defenses. This is one of the stupidest things about the Stellaris AI, IMO; because they mostly build their ships with an imbalance, it's easy for the player to exploit it... and you can predict the imbalance in advance just by looking at the line under the empire name in the Diplomacy screen!

Wait, which way do you want it?!? If the AI just switches to doing whatever best counters the player (or other AI that it's currently fighting, I suppose), then the AIs will obviously just end up doing the same thing that the player is doing - balanced ships in the early game, then going heavy on the deeply overpowered Neutron Launchers later on, never using missiles (except maybe torp 'vettes in the late game) or strike craft, etc. If they instead keep doing their current thing - even if you were to randomize the strategies so that it was actually necessary to scout their ships before countering them - you could (and should) continue to exploit the heck out of it as they "make the same mistakes again and again".

Also, the AI does, in fact, already refit its ships somewhat in response to enemy ships. For example, if you're enough of a fool to put missile batteries on your starbases, the AI will show up packing enough point defense to neutralize every single missile you launch. You usually can't convince the AI to diverge too far from its favored weapon mix - if you start running 100% shields against an AI that you recently saw running 100% lasers, it might bend enough to go 40% kinetics 60% lasers instead - but it does adapt to some degree.

Well, except the various spaceborne aliens and crises (including the Fallen and Awakened Empires). Those never refit their ships, so you can (and should) counter-fit them hard.
Click to expand...

The biggest root problem there bolded. As there shouldn`t be just one overpowered best versus all super weapon type, but different strategies where you could use either mostly beam weapons, missiles or fighters to be effective. (Although that is not to say something like just reading an empire name will automatically give an experienced player a blueprint for the AI behaviour isn`t a big and quite silly problem as well. As you would think it could be quite easily solved by mixing the AI personality components better.) And yeah, if the AI is capable of adapting after it encounters a player trying a certain tactic against it, it would make it feel more intelligent and less artificial. As a good human player is also capable of adapting his/her strategies and tactics to the current situation. (And that is also probably why I am not that good of a player usually as I just like to stick to something that has worked in the past. ;) )
 
MatRopert said:
So by typing 'debug_ai' in the console and observing an AI empire, you can see what it has in mind:
Click to expand...

When I type debug_ai (or switch tags to them) no extra info appears. Do I need to do something else to "observ an AI"? Also the AI doesn't seem to know to combine force with its allies. I'm currently in a hegemonic federation with three other empires. While individually my fleet is smaller than that of a neighbouring devouring hive, the fleet of the combined federation is much larger. Taking my chances, I declared war. Judging from my previous AI behavior in previous patches, I would expect my allies to start following one of my fleets. Instead they do nothing, just sitting over their homeworld. As a consequence the Hive is able to conquer my systems. Until it stops two systems away from my capital. From the Hive perspective that's smart. Moving further would bring it in range of the federation's homeworlds, three worlds with each over 1k fleet power hovering over them. My federal allies are too passive though. If they just combined their fleet (a short trip as we're right next to each other) we could easily crush each other.

So please, examine the old fleet behaviour and make the AI more proactive in following their allies' fleets.
 
Moah said:
You need to be in observer mode, and observing the specific country whose AI you want to analyze.
Click to expand...

Is there a way to observe what its thinking regarding starbases? I noticed in my game, the AI is being completely crippled by the fact that they won't upgrade their starbases and get more anchorages. They all have massive stockpiles of alloy and they're all at their cap in terms of number of starbases and fleets, but they are extremely slow to upgrade actual starbases to build up capacity.
 
semisweet said:
You can make weights irrelevant without needing to make them break mods.
Click to expand...

that seems to be what they did. Have the AI use all available plans first and then use weights. So if you are playing vanilla the AI will usually have plans to work with (up until a lategame point where they are filthy rich anyways) and if your mod still uses weights you can delete all plans and the AI will work as it did before.

I think the AI sometimes running out of plans is not a design goal but due to the fact that the scope of the update seems to have been "build a functioning plan system. Let designers write more complex/encompassing plans after the prototyp has survived contact with players for a while":
 
You must log in or register to reply here.
Top Bottom
Back