• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Stellaris Dev Diary #227 - Looking after the AI

KdHFPIxnbfUdecDUWFOFpKrSq3pNeDU6BQdeWWLLCrYe655fAX8gyF_3548erNqCH1DYaMfK8qb7BwsbIZwsd22bEQ3q9jmM7-jKe4dUCd--WO07urWQEkcuT3b-0pJYzWnHB--e=s0


Hello and welcome to a new Dev Diary,

My name is Guido and today I’m here in my role as a Principal Designer on Stellaris to talk about AI in a bit more detail.

You probably have heard about the Custodian Initiative by now which has been created to keep improving the game on a more regular basis and in order to be quicker when reacting to player feedback. A part of this initiative is also to put some more love and attention to the AI of the game going forward - an AI initiative inside the Custodians, basically.

For this, we have set some goals for ourselves going forward:
  1. Always work on AI-related topics, regardless of what else is going on
  2. Move the AI towards being challenging to players in an entertaining way, rather than be optimized to min-max its way to victory
  3. Move the AI towards being more distinctive, so that not all empires feel strong in the same way
  4. Support future DLCs from the get-go
  5. Constantly make small improvements to the AI
  6. React quicker to player feedback
  7. Occasionally make a push for more significant improvements
Speaking of which, for the upcoming patch in November, we have some significant updates in store.

Economic Script Update​


First of all, the biggest change you will notice is how we have changed the economic plans script. This script is the core of the economic behavior of our empires. It defines what resources they strive to get when building districts and buildings. How much population growth they should go for and how much research and unity they want.

The functionality of the script hasn’t changed much, but how we are using it has changed.

Previously the script was divided into early-, mid-and late-game. Depending on the phase of the game, empires would prioritize resources differently. For example, focus on research was lower in the early game than in the later stages of the game. However, this approach didn’t take into account the various situations an empire can find itself in. Especially after a war or when a new empire breaks off an existing one. In those cases, even if the game phase was in the late game, for the respective empires it meant that they were in a much more ‘early game’ position.

uBhcCRICEVI9OQc0U-dQMkvrUuRI9XEsu0KdS3naOfBmz7zMa3jbIVU5q5zwWYshpObGTaTqau-zMAW1SiRcHXYOCNRw1qvq0zbQrPepOEP9jAi1K0kuezFyEsQ0heKhJ84-idbl=s0
LT2BJzggvYTk6h4XYQCGbLlvBzvXZ2PcUyhl3e5myTpzkO2jLqXdCRhIDrs9oUipqiM4uuh89BdjMiW30PUXU2HrXtE3dsHufnJK0TFiAWjZtmRgB2OKmXRzATX-2kNzRJfofA0p=s0


Instead of having 3 different economic plans, we feature 1 base plan instead. In order to get more flexibility and to react to the empire’s situation, we’re relying much more on the ‘subplans’ inside that base plan.

Improved economic subplans​


Subplans can be turned on or off, depending on the situation the empire finds itself in. Our main rationale was to ensure that an empire would be economically stable before it spends resources on ‘bonus’ things like research, population growth, defensive modules on starbases, and unity buildings.

Previously those things were prioritized too early and without enough respect to the basic income of energy and minerals, leading to empires that produced alloys, but had big deficits in energy and mineral production. And this deficit would be the start of an economic death spiral, where the resource debuffs would further reduce production and everything just escalated to the point where an empire was bankrupt on all resources. This became especially problematic after the economic system has been rebalanced to focus resource production more on the districts, rather than the buildings of a planet.

Here’s an example of what the economic situation generally looked for empires in a game that went on for around 80 years:

JkCSTjSHVsY87vJzUb5vNrO_K8aHiUIs6vu-6AQHjeP2sNF_ImdCKwdRaCpSp6mZrqnBSr3eFpfwdu-eUMG-uN16duNFGgaMqw5Ipm4Fd1V4UtwxN5tuTy2X2mEi6_EC_OODtvFb=s0

(These are screenshots from Stellaris version 3.0.3)

Our updated economic script prioritizes basic income first and takes the new economic rebalance into account. Energy and minerals are most important.

The difference between the ‘income’ and ‘focus’ block is that if the monthly income is below what is defined in the ‘focus’ block - districts and buildings which produce those resources get an extra bonus in weight, when deciding what to build.

basic_plan_02.jpg


Then the first subplan kicks in. If a country uses food (therefore, Machine empires will have this subplan turned off) it will prioritize food production.

subplan_1.jpg


The next subplan will check conditions for focusing on consumer goods. Again, checking if the empire actually uses them or not - and then only focus on producing them if the empire has at least a monthly income of minerals of 30.
Based on the fact that in order to create consumer goods you require minerals.
subplan_2.jpg



Further down we activate the plans for prioritizing research and all the higher-level resources

CS9qyuZNW1naD8IG6k1-5GyF-QkYeF_Vw2k9Lugo6rrBtdcFzksDJ2W6X1AHhjHZi44UUkapwAWkaUFBRkvlJ2VxNJp2GQv-6dwBjjCcv86v-sIuInAnfOPr_zYYJzT88QpS51qi=s0


Resulting behavior improvements​


So, the script can check for various situations in AI empires - from the fact if they are a Gestalt Empire, using food to monthly income of specific resources.
This gives the AI a lot more flexibility in managing its economy.

As an example, here we have a 100-year old Galaxy with 13 AIs and every empire is able to manage its economy in a decent enough way. Notice the resource tab at the top - almost all empires have positive income in all resources; the ones with a negative income only have a small deficit:

97IOcqgoSBd3m6c8CQbtqO5z6Yx4XkWopX5HciBdo_4zqVqepRs4QDnnMysgLyute8GQust0nDCT14UMrrSL38p08pcKWHbrvIMbFA8Aj_9WTVpfWVVo8Tw-Qpft6Fi9uJPaSA-c=s0


Apart from this, there were some small, but significant code changes that helped the AI in running the show.

Conclusion​


The code for the AI has been optimized heavily in the past in order to improve performance a lot. However, this has led to some unforeseen and unintended behaviors which have now been corrected. Some of the districts and buildings weren’t considered at all and city districts were weighted way too high. The AI is also now able to build temples and holo theaters, for example.

Finally, the AI has also been given a bit of support in how it will set up its starbases, especially in conjunction with the hydroponics starbase building, which can play a larger role in how you provide food for your empire. The AIs can now use more varied setups when building their starbases, making use of Curator Think Tanks, Nebula Refineries, and other special buildings where it makes sense.

And all of this was built on the foundation of the last major rework of the economic AI, so kudos to @sidestep for making this evolutionary step possible.

With your help, we’re looking forward to giving the AI the attention it deserves and making it even better in the future.

Cheers,
Guido
 
Last edited:
  • 158Like
  • 70Love
  • 14
  • 7
  • 3Haha
  • 1
Reactions:
  1. Move the AI towards being challenging to players in an entertaining way, rather than be optimized to min-max its way to victory
  2. Move the AI towards being more distinctive, so that not all empires feel strong in the same way

I am not sure how exactly the team is planning to make the AIs more distinct but here are couple thoughts which came to my mind. Let's take a look of couple different kinds of empires and how they could behave to make them different and how it relates to the economy AI.
  • Spiritualist empire might want to spread their faith (ethics) to other empires. Depending on the other ethics they could do so by conquering others or by forming diplomatic ties. Currently there is no distinct way to do that by diplomatic ties though doing diplomacy with a spiritualist empire increases the faction attractiness. In either case the Spiritualist Empire needs a strong economy and military to either conquer the other empires or make itself more attractive for the diplomatic relations.
  • Militarist empire wants to conquer and fight against the rivals. It needs a strong economy to wield a strong military to do that.
  • Pacifist empire wants to be left alone. To avoid getting attacked by the stronger empires it needs a strong economy and military to discourage that (and if not Xenophobe to make itself more attractive for diplomatic treaties).
At this point you probably can see what every empire needs no matter what their ultimate goals or preferences might be. This leads to my thought that trying to make "roleplaying" economy just leads to a stagnant AI and any "roleplaying" aspects should be handled by the other mechanics the game provides. Clerks might be fitting for a Megacorp empire but if the planet needs amenities they are very poor way of getting them and the AI should build that holo-theater.

Obviously civics which modify how the economy works (eg. catalytic processing) should change how the AI attempts to create a strong economy.

Currently the game has surprisingly many ways to make the empires behave differently but unfortunately AI doesn't really use them. Ascension perks are probably the most obvious example. For example when it comes to Ascension Perks AI (from what I have observed) tends to pick one soon as they can which leads them to picking up random perks which don't really change how the empire behaves.

Instead of picking randomly the AI empires should have some long term plans. For example:
  • Remants origin decides it wants to restore the relic homeworld and that they don't need Arcology perk.
  • Another empire discoveres a relic world and decides the same.
  • Third empire empire decides it can turn a large mineral and energy poor planet into a city planet and saves a slot for the Arcology perk.
  • Spiritualist empire decides it wants to pick Psionic ascension so it knows to save two slots for them until it has researched the required tech.
  • Xenophile empire wants to turn all planets into paradises so it saves one slot for the World-Shaper.
  • Empire discovered a black hole system and decides it wants to build a matter decompressor there. It saves required slots for Galactic Wonders and maybe Master-Builders.
  • Each of these would require the AI also to stockpile required resources to they can actually act on the plans.
AI behaviour in wars is also pretty similar no matter if you are fighting a genocidal or fanatic pacifist empire so that leaves lot of room to make the empires fight differently. For example a pacifist empire in a defensive war might prefer to stay on the defensive and only attack to conquer or occupy nearby bottlenecks, enemy systems near colonized planets to give extra warning time if a fleet approaches etc. and would otherwise prefer to stay on defensive unless outnumbering the enemy. Meanwhile a militarist empire might be more aggressive, attempting to seize bottlenecks deeper in the enemy territory or trying to attack enemy colonies more aggressively.

Short summary: no matter what kind of behaviour the AI empires have they all need a strong economy and there is only so many good ways to achieve that. Trying to roleplay the AI economy is likely to create complicated AI rules which makes the actual economy inefficient and even if the economy is different it is not really visible to the player. Poor economy created by such also makes it harder for the AI to act differently in ways which are actually visible to the player.
 
  • 4
  • 3Like
Reactions:
Does/can this system take into account AI ethics/personalities as well? It would make for a more varied and interesting game if, once its basic needs are met, a militarist empire had a higher base focus on minerals and alloys, letting it support bigger fleets than a similarly sized but more materialist focused empire that would then spend a higher base proportion of its efforts on research. Spiritualists/Pacifists should be more likely to spend on food/pop growth, Egalitarians more consumer goods/amenities focused, etc.
 
  • 2Like
  • 2
Reactions:
I always like to see the AI being given the ability to deal with less straightforward features.

One of the things that has still been missing is for psionic AI empires to interact with the Shroud, create avatars, form covenants, etc.

Is this something that could one way find its way into the game?
 
Funny this is exactly what I thought was lacking with all the economic AI systems in Stellaris.
Previously the same initiative should have lead to huge conditional list being included with buildings so the AI has some guidelines when to build what. The system existed and I am sure it was the Gladius AI mod back then which made extensive use of this to improve the AI, but there was absolutely nothing from Pdx in that regard.
It's the same with the new system until now, you had a great basis to teach the AI to play, but nobody at PDX bothered to do so and even worse because nobody did this you couldn't discover all the tiny bugs that might have existed in the source code which modders could neither see, touch or make a ticket(that will be heeded) to the engine department.

Thank you for finally USING the actual systems you guys created to tune the AI.
 
Does this AI demolish and rebuilds buildings?

Because the problem is not just deciding the targets, but deciding what to tear down. And if an AI builds/gets a Dyson sphere, will it respec the planets out of generator worlds for more research?

This is extremely important for AI that conquers. Most of the time their conquests are random botched planets.
 
  • 1
Reactions:
One addition is also needed on that list:

Keep an eye on Crisis AI, because it has been so broken in the past, and it was always the most fragile thing on any? DLC release.
 
  • 2
  • 1Like
Reactions:
I am not sure how exactly the team is planning to make the AIs more distinct but here are couple thoughts which came to my mind. Let's take a look of couple different kinds of empires and how they could behave to make them different and how it relates to the economy AI.
  • Spiritualist empire might want to spread their faith (ethics) to other empires. Depending on the other ethics they could do so by conquering others or by forming diplomatic ties. Currently there is no distinct way to do that by diplomatic ties though doing diplomacy with a spiritualist empire increases the faction attractiness. In either case the Spiritualist Empire needs a strong economy and military to either conquer the other empires or make itself more attractive for the diplomatic relations.
  • Militarist empire wants to conquer and fight against the rivals. It needs a strong economy to wield a strong military to do that.
  • Pacifist empire wants to be left alone. To avoid getting attacked by the stronger empires it needs a strong economy and military to discourage that (and if not Xenophobe to make itself more attractive for diplomatic treaties).
At this point you probably can see what every empire needs no matter what their ultimate goals or preferences might be. This leads to my thought that trying to make "roleplaying" economy just leads to a stagnant AI and any "roleplaying" aspects should be handled by the other mechanics the game provides. Clerks might be fitting for a Megacorp empire but if the planet needs amenities they are very poor way of getting them and the AI should build that holo-theater.

Obviously civics which modify how the economy works (eg. catalytic processing) should change how the AI attempts to create a strong economy.

Currently the game has surprisingly many ways to make the empires behave differently but unfortunately AI doesn't really use them. Ascension perks are probably the most obvious example. For example when it comes to Ascension Perks AI (from what I have observed) tends to pick one soon as they can which leads them to picking up random perks which don't really change how the empire behaves.

Instead of picking randomly the AI empires should have some long term plans. For example:
  • Remants origin decides it wants to restore the relic homeworld and that they don't need Arcology perk.
  • Another empire discoveres a relic world and decides the same.
  • Third empire empire decides it can turn a large mineral and energy poor planet into a city planet and saves a slot for the Arcology perk.
  • Spiritualist empire decides it wants to pick Psionic ascension so it knows to save two slots for them until it has researched the required tech.
  • Xenophile empire wants to turn all planets into paradises so it saves one slot for the World-Shaper.
  • Empire discovered a black hole system and decides it wants to build a matter decompressor there. It saves required slots for Galactic Wonders and maybe Master-Builders.
  • Each of these would require the AI also to stockpile required resources to they can actually act on the plans.
AI behaviour in wars is also pretty similar no matter if you are fighting a genocidal or fanatic pacifist empire so that leaves lot of room to make the empires fight differently. For example a pacifist empire in a defensive war might prefer to stay on the defensive and only attack to conquer or occupy nearby bottlenecks, enemy systems near colonized planets to give extra warning time if a fleet approaches etc. and would otherwise prefer to stay on defensive unless outnumbering the enemy. Meanwhile a militarist empire might be more aggressive, attempting to seize bottlenecks deeper in the enemy territory or trying to attack enemy colonies more aggressively.

Short summary: no matter what kind of behaviour the AI empires have they all need a strong economy and there is only so many good ways to achieve that. Trying to roleplay the AI economy is likely to create complicated AI rules which makes the actual economy inefficient and even if the economy is different it is not really visible to the player. Poor economy created by such also makes it harder for the AI to act differently in ways which are actually visible to the player.
well, you miss one thing, those points they want to work on and you talk about arent points around economic ai but overall ai, they sayed this just to tell us what they want to reach with further development of other ai parts if they finished economic ai
 
My goodness, well this is all good stuff. But seeing now the completely logical way that the AI is mapped out for these changes, I can't help but feel really frustrated that 2.0 has been out for, what, almost 3 years? And this restructuring is happening now? Admittedly I've been away from Stellaris for a while, but good grief guys. I'm glad you're getting to it, but why did it take this long to make fundamental economy management a thing that the AI could do?
 
  • 2
Reactions:
it would be cool if they were subplan for different civic like catalytic processing

You mean like this? ;)

(which is in Lem BTW)

1633638110397.png



Will the AI improvements also make sector governors better if we try to use them instead of micromanaging everything ourselves?

The sector AI, planetary automation AI and AI economic minister are all different systems, so no these changes won't change the former two.
 
  • 13Like
  • 6
  • 1Love
Reactions:
My goodness, well this is all good stuff. But seeing now the completely logical way that the AI is mapped out for these changes, I can't help but feel really frustrated that 2.0 has been out for, what, almost 3 years? And this restructuring is happening now? Admittedly I've been away from Stellaris for a while, but good grief guys. I'm glad you're getting to it, but why did it take this long to make fundamental economy management a thing that the AI could do?
The AI got an overhaul in 2.6, and had improvements done in 2.8 and 3.1 (memories are foggy, so excuse me if I get version numbers wrong or miss one). I would hardly say it's something we haven't been working on.

With a game as complicated as Stellaris, it takes many iterations to get something to a point where it's solid enough to release, experiments produce unexpected results, etc.

Will the AI improvements also make sector governors better if we try to use them instead of micromanaging everything ourselves?

Planetary automation and sector automation use a different system than the empire AI.

I'm a single issue voter. I see no megacorp, I hit the red button.

Only slightly biased, right? :D
 
  • 2
  • 1Like
  • 1
Reactions:
This sounds like an excellent foundation, but I have one concern - currently it's extremely easy for the player to get ahead in research, which results in getting ahead in resources and military development, and an early tech advantage often snowballs such that you're a clear step up from everyone else in the galaxy by midgame. If the AI now has even less focus on research and other advanced outputs, will this problem be exacerbated?
 
  • 1
  • 1Like
Reactions:
This sounds like an excellent foundation, but I have one concern - currently it's extremely easy for the player to get ahead in research, which results in getting ahead in resources and military development, and an early tech advantage often snowballs such that you're a clear step up from everyone else in the galaxy by midgame. If the AI now has even less focus on research and other advanced outputs, will this problem be exacerbated?

The opposite happens in fact! Since the AI is now better at building up their basic economy, they are in turn better at getting research up and running. Granted the default difficult setting won't be as good as a min-maxing player, but on higher difficult levels it should be more challenging than it is currently.
 
  • 14
  • 2Like
Reactions:
What does the AI do once it achieves its goals? If I understand these right, an income value in a plan is a target income, and once it achieves that, the plan is fulfilled, and it goes on to try and satisfy the next income target down the list? What happens when it gets everything in the green? Surely the AI isn't simply happy with 30 alloys of income for all time?
 
  • 1Like
Reactions:
What does the AI do once it achieves its goals? If I understand these right, an income value in a plan is a target income, and once it achieves that, the plan is fulfilled, and it goes on to try and satisfy the next income target down the list? What happens when it gets everything in the green? Surely the AI isn't simply happy with 30 alloys of income for all time?

The final sub-plan that we're currently looking at is set up as follows:

Code:
@consumer_goods_target = 30
@research_target = 9999
@unity_target = 70
@alloy_target = 500
@rare_resource_target = 4

income = {
    physics_research = @research_target
    society_research = @research_target
    engineering_research = @research_target
    unity = @unity_target
    alloys = @alloy_target
    exotic_gases = @rare_resource_target
    volatile_motes = @rare_resource_target
    rare_crystals = @rare_resource_target
}
 
  • 15
  • 3Like
Reactions: