• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Stellaris Dev Diary #227 - Looking after the AI

KdHFPIxnbfUdecDUWFOFpKrSq3pNeDU6BQdeWWLLCrYe655fAX8gyF_3548erNqCH1DYaMfK8qb7BwsbIZwsd22bEQ3q9jmM7-jKe4dUCd--WO07urWQEkcuT3b-0pJYzWnHB--e=s0


Hello and welcome to a new Dev Diary,

My name is Guido and today I’m here in my role as a Principal Designer on Stellaris to talk about AI in a bit more detail.

You probably have heard about the Custodian Initiative by now which has been created to keep improving the game on a more regular basis and in order to be quicker when reacting to player feedback. A part of this initiative is also to put some more love and attention to the AI of the game going forward - an AI initiative inside the Custodians, basically.

For this, we have set some goals for ourselves going forward:
  1. Always work on AI-related topics, regardless of what else is going on
  2. Move the AI towards being challenging to players in an entertaining way, rather than be optimized to min-max its way to victory
  3. Move the AI towards being more distinctive, so that not all empires feel strong in the same way
  4. Support future DLCs from the get-go
  5. Constantly make small improvements to the AI
  6. React quicker to player feedback
  7. Occasionally make a push for more significant improvements
Speaking of which, for the upcoming patch in November, we have some significant updates in store.

Economic Script Update​


First of all, the biggest change you will notice is how we have changed the economic plans script. This script is the core of the economic behavior of our empires. It defines what resources they strive to get when building districts and buildings. How much population growth they should go for and how much research and unity they want.

The functionality of the script hasn’t changed much, but how we are using it has changed.

Previously the script was divided into early-, mid-and late-game. Depending on the phase of the game, empires would prioritize resources differently. For example, focus on research was lower in the early game than in the later stages of the game. However, this approach didn’t take into account the various situations an empire can find itself in. Especially after a war or when a new empire breaks off an existing one. In those cases, even if the game phase was in the late game, for the respective empires it meant that they were in a much more ‘early game’ position.

uBhcCRICEVI9OQc0U-dQMkvrUuRI9XEsu0KdS3naOfBmz7zMa3jbIVU5q5zwWYshpObGTaTqau-zMAW1SiRcHXYOCNRw1qvq0zbQrPepOEP9jAi1K0kuezFyEsQ0heKhJ84-idbl=s0
LT2BJzggvYTk6h4XYQCGbLlvBzvXZ2PcUyhl3e5myTpzkO2jLqXdCRhIDrs9oUipqiM4uuh89BdjMiW30PUXU2HrXtE3dsHufnJK0TFiAWjZtmRgB2OKmXRzATX-2kNzRJfofA0p=s0


Instead of having 3 different economic plans, we feature 1 base plan instead. In order to get more flexibility and to react to the empire’s situation, we’re relying much more on the ‘subplans’ inside that base plan.

Improved economic subplans​


Subplans can be turned on or off, depending on the situation the empire finds itself in. Our main rationale was to ensure that an empire would be economically stable before it spends resources on ‘bonus’ things like research, population growth, defensive modules on starbases, and unity buildings.

Previously those things were prioritized too early and without enough respect to the basic income of energy and minerals, leading to empires that produced alloys, but had big deficits in energy and mineral production. And this deficit would be the start of an economic death spiral, where the resource debuffs would further reduce production and everything just escalated to the point where an empire was bankrupt on all resources. This became especially problematic after the economic system has been rebalanced to focus resource production more on the districts, rather than the buildings of a planet.

Here’s an example of what the economic situation generally looked for empires in a game that went on for around 80 years:

JkCSTjSHVsY87vJzUb5vNrO_K8aHiUIs6vu-6AQHjeP2sNF_ImdCKwdRaCpSp6mZrqnBSr3eFpfwdu-eUMG-uN16duNFGgaMqw5Ipm4Fd1V4UtwxN5tuTy2X2mEi6_EC_OODtvFb=s0

(These are screenshots from Stellaris version 3.0.3)

Our updated economic script prioritizes basic income first and takes the new economic rebalance into account. Energy and minerals are most important.

The difference between the ‘income’ and ‘focus’ block is that if the monthly income is below what is defined in the ‘focus’ block - districts and buildings which produce those resources get an extra bonus in weight, when deciding what to build.

basic_plan_02.jpg


Then the first subplan kicks in. If a country uses food (therefore, Machine empires will have this subplan turned off) it will prioritize food production.

subplan_1.jpg


The next subplan will check conditions for focusing on consumer goods. Again, checking if the empire actually uses them or not - and then only focus on producing them if the empire has at least a monthly income of minerals of 30.
Based on the fact that in order to create consumer goods you require minerals.
subplan_2.jpg



Further down we activate the plans for prioritizing research and all the higher-level resources

CS9qyuZNW1naD8IG6k1-5GyF-QkYeF_Vw2k9Lugo6rrBtdcFzksDJ2W6X1AHhjHZi44UUkapwAWkaUFBRkvlJ2VxNJp2GQv-6dwBjjCcv86v-sIuInAnfOPr_zYYJzT88QpS51qi=s0


Resulting behavior improvements​


So, the script can check for various situations in AI empires - from the fact if they are a Gestalt Empire, using food to monthly income of specific resources.
This gives the AI a lot more flexibility in managing its economy.

As an example, here we have a 100-year old Galaxy with 13 AIs and every empire is able to manage its economy in a decent enough way. Notice the resource tab at the top - almost all empires have positive income in all resources; the ones with a negative income only have a small deficit:

97IOcqgoSBd3m6c8CQbtqO5z6Yx4XkWopX5HciBdo_4zqVqepRs4QDnnMysgLyute8GQust0nDCT14UMrrSL38p08pcKWHbrvIMbFA8Aj_9WTVpfWVVo8Tw-Qpft6Fi9uJPaSA-c=s0


Apart from this, there were some small, but significant code changes that helped the AI in running the show.

Conclusion​


The code for the AI has been optimized heavily in the past in order to improve performance a lot. However, this has led to some unforeseen and unintended behaviors which have now been corrected. Some of the districts and buildings weren’t considered at all and city districts were weighted way too high. The AI is also now able to build temples and holo theaters, for example.

Finally, the AI has also been given a bit of support in how it will set up its starbases, especially in conjunction with the hydroponics starbase building, which can play a larger role in how you provide food for your empire. The AIs can now use more varied setups when building their starbases, making use of Curator Think Tanks, Nebula Refineries, and other special buildings where it makes sense.

And all of this was built on the foundation of the last major rework of the economic AI, so kudos to @sidestep for making this evolutionary step possible.

With your help, we’re looking forward to giving the AI the attention it deserves and making it even better in the future.

Cheers,
Guido
 
Last edited:
  • 158Like
  • 70Love
  • 14
  • 7
  • 3Haha
  • 1
Reactions:
Could you give us a way to see how the player empire is doing compared to the AI plans?

When I see a income of 40 Food, it is hard to say if that is barely enough or a massive abundance for my empire at this part of the game.
Shortages in some materials might "sneak up" on you, as +40 Minerals with a large storage still looks solid.
A simple yellow color + the target value shown in tooltip could be enough.

This would also be a assistance for AI developers, as they can quickly spot where the AI is falling behind the plan in resources.
I would consider +40 food as "always too much"; I rarely go above +5 unless my stockpile is comically low, the rest gets sold as a monthly trade on the market. Likewise, +40 Minerals is always low considering that's 10 months to support building one building/district. While the need for Minerals obviously goes down as the game goes on, you should always try and maintain a decent (100-200) income (post-trades) to support the more expensive Mineral use-cases (Ecunopoli, etc.). Maybe
The the entire economy crashes, because it has not enough minerals to run Alloy or CG production and it is in a death spiral.
Or becuase food becomes a shortage, and there is massive loss of Happiness. Wich ends in a daeth spiral.
The issue was a lack of the most basic resources, which collapsed all building and maintenance ability.

The best your idea would to is build more things it can not support and then have even less population and stored minerals to fix the shortages that will bring it down.
First off, the AI should have been able to handle the above. It should know it needs to build more Mining Districts in order to support more Industrial ones.

By flipping the logic to focus on input resources first, you solve this one but also potentially introduce others. As I noted, you now run the risk of the AI stockpiling basic resources beyond what it needs, which would prevent the AI from truly ever being a threat.

The REAL problem is you can't ever tell the AI what to fo for every possible use case manually; its too many cases to consider. What you need is a way to tell the AI what resources to maximize, a way to define the relationships between the various resources, and a game engine smart enough to tell the AI "you need more mining districts to support gaining more alloys stupid".

That's the real problem. What I see is an improvement, but still a bandaid.

*is a software engineer
 
  • 1
Reactions:
I would consider +40 food as "always too much"; I rarely go above +5 unless my stockpile is comically low, the rest gets sold as a monthly trade on the market. Likewise, +40 Minerals is always low considering that's 10 months to support building one building/district. While the need for Minerals obviously goes down as the game goes on, you should always try and maintain a decent (100-200) income (post-trades) to support the more expensive Mineral use-cases (Ecunopoli, etc.). Maybe
And seeing the AI plan would help me do exactly that. Which is why I asked for this.

First off, the AI should have been able to handle the above. It should know it needs to build more Mining Districts in order to support more Industrial ones.
So in other words, it should focus on basic resources before it consumes them into Advanced ones? Exactly what this DD talks about?

Well it seems like you do agree with what was said, so why do you sound like you disagree?
 
  • 1
Reactions:
And seeing the AI plan would help me do exactly that. Which is why I asked for this.


So in other words, it should focus on basic resources before it consumes them into Advanced ones? Exactly what this DD talks about?

Well it seems like you do agree with what was said, so why do you sound like you disagree?
Implementation matters; I'm worried the devs will fix basic income shortages, but in a way that prevents the AI from spending those resources.

I've made systems similar yo what Stellaris has, and in my experience you will never have a fully functioning AI by trying to micro-manage (via plans) what the AI will do. You'll always be trying to triage one issue after another, and never really have a system to handle every use case.

Simple example: the AI still won't be able to handle replacing district's. Or changing a planet's build order as other planet's get colonized/captured. That requires significantly more work then we're getting in this update.

What I want, is rather trying to shuffle the plans around, is to make a system that actually works.
 
  • 1Like
Reactions:
I'd like to suggest not dismissing the idea of AI min-maxing. One of the few things the AI can best a player at is collecting data and crunching numbers. The Galactic market is the obvious example. The AI could easily keep track of the real cost of resource inputs verses the value of output and find opportunities to correct the market and profit from it. Things like we're to place ships is a hard thing to determine, especially with incomplete information, but "is X > Y" is a question the AI is perfect for.

Arbitrage FTW!
 
I'd like to suggest not dismissing the idea of AI min-maxing. One of the few things the AI can best a player at is collecting data and crunching numbers. The Galactic market is the obvious example. The AI could easily keep track of the real cost of resource inputs verses the value of output and find opportunities to correct the market and profit from it. Things like we're to place ships is a hard thing to determine, especially with incomplete information, but "is X > Y" is a question the AI is perfect for.

Arbitrage FTW!
Right, but the problem is you then have all the AIs trying to do the same thing at the same time destabilizing the market every few months. So you need an alternative then just hard-coding the AI to "purchase/sell when price reaches some threshold". And you need different AIs to have different needs/wants so they don't trash the market.

Hence why AIs get so comically hard.
 
I'd like to suggest not dismissing the idea of AI min-maxing. One of the few things the AI can best a player at is collecting data and crunching numbers. The Galactic market is the obvious example. The AI could easily keep track of the real cost of resource inputs verses the value of output and find opportunities to correct the market and profit from it. Things like we're to place ships is a hard thing to determine, especially with incomplete information, but "is X > Y" is a question the AI is perfect for.

Arbitrage FTW!
I don't think a lot of playes would like that tbh

Like AI was already beaten player in buying slave cuz they are literally faster than player can even open the market, this led to player think it is unfair thus they got restricted on how they buy slave now.

So do understand that when people said they want good/competent ai, they means good/competent ai that they can beat like human player, not ai that take advantage of them being ai like being faster than human can physically clicking, insane number crunching capability, etc.
 
  • 2
Reactions:
Right, but the problem is you then have all the AIs trying to do the same thing at the same time destabilizing the market every few months. So you need an alternative then just hard-coding the AI to "purchase/sell when price reaches some threshold". And you need different AIs to have different needs/wants so they don't trash the market.

Hence why AIs get so comically hard.
Not necessarily, if the AI is calculating its individual modifiers. An empire with mining guilds civic has lower costs for getting minerals and is less likely to buy them, while a weak and wasteful race is quicker to reach a point where minerals are costly to produce. Plus, any change in energy production affects the relative value of everything.
Furthermore, there are a ton of small modifiers to production and consumption, which means it's unlikely that two empires will have the exact same costs associated with resources, if they're calculating the finer details. If they're not doing that detail then obviously it won't work. If we're assuming it won't be done well then it's not going to work, but if done well, it could work.