Stellaris Dev Diary #172 - Reworking the AI

[sidestep]

Well, it was a good read. Better than most DDs by far, especially because follow-up questions and concerns are being addressed.
Also, glad you finally found some time to finish necessary debug tools and steamline some AI code (which should have been there at release, to be honest >.>) - hopefully, this will make further AI adjustments faster and easier.


One thing I did not understand though: do plans adjust for difficulty? As I understand it, when AI income is doubled, the "normal" plans stop being an effective motivation. While AIs could, theoretically, switch to the next plan, the pop requirements will, likely, keep AIs back.

Yeah, having them just switch over to "the next" plan is something I would like, but if they have a really solid economy then falling back to the old way of building until they hit the next plan isn't too bad in my experience.

 

[Liggi]

Really blown away by these improvements, and all the questions you're answering here. My hype level is through the roof.

 

[sidestep]

In Operations Research (the science of optimisation) you get around the general issue you describe by using a value which is practically infinite. A cost which is prohibitively high, a value whichis excessively high.

E.g. if all other weights are 1, then 10,000 should do the trick.

In the practical gaming context, a target of 10,000 alloys would encourage the AI to produce as much as it possibly could.

Also have the developers thought about applying control theory to the AI

This is true, but in this case this would also mean that the AI only produces alloys since that score will now always outweigh all others. But setting a latelatelate-game+ plan to just be alloys and science 10.000 would probably be a fine approximation of the behavior you want

 

[sidestep]

Is every AI "minister" behaving the same whatever their ethics, civics and personality?

It would be interesting to vary weighs based on those, that would provide variability. If you are militaristic a more offensive strategy, or a more foundry based economy could do.

I find ai too predictable. I know what a fleet will do, how will it move.. sometimes, just a more random approach will surprise the human, not just because is better or worst, but because the ai strategy is different. Even mixing stupid strategies and good scripted ones would add a surprise flavour.

For example, I know that ai will not position its fleets on the border before attack, so DOWs give you some time for preparations if their capital is far. I know ai doesn't care about civilian ships, so putting a construction/science ship in the side of a system gives you intel deep in uncontrolled territory. I know ai fleets get stuck on heavily fortified worlds, bombing it into submission. I know ai doesn't repair their fleets on captured stations..... i don't mean this is wrong. Maybe that is the optimal approach, but maybe 10% of the times, ai should do different things to surprise the human. Usually gamers are impressed when an ai does something unexpected and think the ai is being smart... when the thing is that it's just doing something not so very predictable. (Also wrong strategies are fine, an ai should be "smart" enough to make mistakes sometimes)

No, there are some differences in different ministers depending on personality and such, for example the war AI allocates a lot more offensive fleets if it has an aggressive type of personality. There could probably be more of it, thats true.

 

[FinellyTrained]

So, something that's important to remember is that focus goals are supplementary to the income goals, you need to look at the base income goals as well since they determine a lot of the AI score, but also that goals are just that goals.

Supplementary as in what, to be ignored? My initial understanding is that AI does focus goals first, then goes to "goals" goals. How do they work exactly, like if AI doesn't have 50 net alloys, will it start building research, because research has 200 goal, despite 0 focus.
How does it exactly choose, if current net minerals are 7 and alloys 12?

The early game AI might not even reach 50 alloys, but having that goal higher means that they focus it early ( which is important ) but also that its weighted higher relative to the other goals.

How exactly? I can digest formulas or even code.

I also don't think that 50 alloy income is very high tbh

Then you probably have done some calculations. Default settings early game is 1200 months, say average would be 30. 36k alloys where would they theoretically go?
Forget 100 years, let's talk first 20. 240*30=7200 alloys. What are they going to be spent on?

if you're playing wide early then not being able to build ships and starbases because of lack of alloys is gonna suck.

I don't mind wide as the default, though newbees will actually suffer from opression. Are you sure it the right choice from the sales prospective?
Anyway, how many ships, how many starbases, what are expectations for the speed of advancement? First 20 years, again. Corvettes limited to pretty much 20, bases to pretty much 4. Outposts are limited by +3 monthly influence.

Since focus goals cannot be dynamically set by number of planets the food income focus is there mostly to act as a deficit buffer and make sure that the AI starts thinking about food seriously before it actually hits the negatives. But if we could set it as a multiple of planets that would be neat.

That explains it. Modders gonna suffer. If possible include the condition to choose the plan by the number of planets/pops. Probably modders could just generate couple of hundreds of plans if they need to.

As for the minerals you need to once again look at the base income goal too, the focus goal of 30 acts a lot as a "minimum" income threshold to make sure that the alloy focus goal does not give alloys to high of a score before you have some kind base mineral income. The AI is gonna build a lot more mineral production districts than the focus goal dictates since the base income goal is 200, by far the highest income goal in the early game, giving minerals a very high relative score.

And minimum of 50 alloys requires 9 foundries. 9 additional building slots, 45 additional pops. It is fine for 100 years, not for 20.

The consumer goods focus goal is much like the food one, to make sure that there is a small buffer before going into deficit. If every time the AI built a Science Lab ( or another building requiring consumer goods upkeep ) it would go into deficit then it would just be acting way to reactively. Having a small consumer goods buffer just makes it more stable. Also, yes, the AI will sell consumer goods if it wants/haves to.

I know how Stellaris works, I have played it. Or it may solve it with trade or use stockpiled resources for awhile. Or just disable some jobs, that do not have enough raw materials. Or can just outright cheat the extra resources out of thin air. Solutions may be numerous.
If focus goals are aimed to maintain minimums necessary to prevent negative in case something consuming those resources gets build, then alloys need not to be there at all and cg will have to require some additional rule, since it is both a product and a raw material.

Lastly energy is the easiest resource for the AI to acquire outside of planet construction, it can build mining stations to get more but also sells resources to keep a pretty nice energy stockpile for such things as blockers and leaders. This means that once again that the energy focus goal is more of a buffer than anything else. During our tests the AI had much less issues with energy than anything else, especially since it produced at lot of alloys that it could sell if necessary

If AI does trade, it probably has to be a set of rules of its own. Like AI can easily sell all net food, once it has 1k*planets. Very smart version would probably even understand that selling food is better than building energy districts in early game. But this is probably an overkill.

 

[Make Victoria 3]

This is true, but in this case this would also mean that the AI only produces alloys since that score will now always outweigh all others. But setting a latelatelate-game+ plan to just be alloys and science 10.000 would probably be a fine approximation of the behavior you want

As long as the focus/surplus goal is set even higher than the plan (to only build science and alloy) would work well (even in not latelatelate game).

 

[M@ni@c]

Good quiz question! Honestly, I don't know. But if this is an issue I have no doubt that our QA ( or the community ) will find it and it will become a JIRA for the team to fix.

FYI, my quiz question wasn't hypothetical. I regularly saw this happen in Stellaris before I modded in a fix. So I can pretty much guarantee this will keep occuring if you didn't take this into account in your AI rework.

I have no idea, but game development is a constant give and take and I have no doubt that the team will work to make the game even better no matter what!

In that case I hope you'll keep in mind the idea to set the desired surplus in the strategic resources file rather than by a central plan.

The scoring system does not use the base building weights, they are only used when the plan system is not used.

I hope you can confirm if I'm understanding things correctly.

Am I understanding correctly that if the surplus goals of the plan are met, the AI will switch back to looking at the current system of weights?

Does the AI completely ignore weights until that point?

For instance suppose the AI is currently producing 100 physics research and 100 society research, while the goal is to reach 200 of both. Suppose I have two research buildings, one that provides physics research, the other society. Suppose there's a planet that has a +40% society research modifier, and I have given a weight to the society research building to make it more likely to be built on planets with that research modifier. Will the AI actually look at that weight, or will it simply look for any research building to meet its central plan, meaning there's a 50% chance for it to build the physics building or society building?

 

[Dark_wing]

These changes look great!
Looking forward to the patch!

A few questions:

Are there plans to expand this system to allow for more abstract goals for the AI?
It would be great if you could set something like "have X Fleet power in year Y", "have tech/ascension Peek X in year Y" "be superior in tech to rival X in year Y" etc. and the AI checks and tries to adjusts alloy/unity/tech production.

Can the AI change plans dynamically based on events? (e.g. a crisis spawns or war is declared and the AI switches to a plan with higher alloy production?) Would something like this be scriptable?

Is there a has_ai_plan condition?

 

[sidestep]

@sidestep, @MatRopert thank you for those great dev diaries and for sticking around to answer questions! Will you be continue to work on the Stellaris AI, or do you have to move on to other projects?

Also regarding The Plan, will the goals for the AI be adjusted according to difficulty levels? An AI with 200% income bonus will reach those goals much quicker, so I think the goals should be higher, right? Likewise an AI with -25% income might need lower goals, also to go a bit easier on new players. Is this currently implemented or are you planning to implement something like this? It would be a great way to make the difficulty slider more meaningful.

The Plan does not take difficulty into account, no. But I'm thinking that you could probably just scale the plan numbers by the income bonuses the AI gets with higher difficulty.

@MatRopert will continue working hard on Stellaris ( probably more on AI ), I unfortunately have switched project But I am leaving it in the very competent hands of The French Paradox and the rest of the team, they're great and will do just fine without me!

 

[sidestep]

Funny that you mention EUIV, because it's not only the game I play most but it will also get better Economy-AI in 1.30. However, if I got that correctly, EU4 economy AI will have a rather different design approach from stellaris economy AI, e.g. to my knowledge it will not have any "plans". What I would like to know is

• How much cross-semination of ideas between games do you perceive to be doable?
• What are the reason you are doing things differently between EU4 and Stellaris?
• Will any of the lessons learned while working on Stellaris AI feed into improving EU4 AI?
• And of course: which AI do you consider more competent at the moment - Stellaris or EU4?

Thanks!

We do have regular AI meetings between projects where we share ideas and implementations etc. so we do learn a lot from eachother! As for why we do it differently between EU4 and Stellaris well that's just because they're two very different games under the hood.

 

[AlphaAsh]

AlphaGo Zero.

That's not how you spell my nick

 

[TempusxX]

will this fix AI's not colonizing habitats, I've had games where an AI builds habitats but never colonizes them

 

[Verx90]

So evry AI as a objective , how does it work for war with allies ? When they decide that is better to fight with an allied fleet or to go alone ? How itbwork with an human player ?

 

[Verx90]

will this fix AI's not colonizing habitats, I've had games where an AI builds habitats but never colonizes them

They answered that 10 pages ago , I suggest using "only dev response" .

Anyway , it was addresed and kinda resolved .

 

[sidestep]

Supplementary as in what, to be ignored? My initial understanding is that AI does focus goals first, then goes to "goals" goals. How do they work exactly, like if AI doesn't have 50 net alloys, will it start building research, because research has 200 goal, despite 0 focus.
How does it exactly choose, if current net minerals are 7 and alloys 12?



How exactly? I can digest formulas or even code.


Then you probably have done some calculations. Default settings early game is 1200 months, say average would be 30. 36k alloys where would they theoretically go?
Forget 100 years, let's talk first 20. 240*30=7200 alloys. What are they going to be spent on?



I don't mind wide as the default, though newbees will actually suffer from opression. Are you sure it the right choice from the sales prospective?
Anyway, how many ships, how many starbases, what are expectations for the speed of advancement? First 20 years, again. Corvettes limited to pretty much 20, bases to pretty much 4. Outposts are limited by +3 monthly influence.



That explains it. Modders gonna suffer. If possible include the condition to choose the plan by the number of planets/pops. Probably modders could just generate couple of hundreds of plans if they need to.



And minimum of 50 alloys requires 9 foundries. 9 additional building slots, 45 additional pops. It is fine for 100 years, not for 20.



I know how Stellaris works, I have played it. Or it may solve it with trade or use stockpiled resources for awhile. Or just disable some jobs, that do not have enough raw materials. Or can just outright cheat the extra resources out of thin air. Solutions may be numerous.
If focus goals are aimed to maintain minimums necessary to prevent negative in case something consuming those resources gets build, then alloys need not to be there at all and cg will have to require some additional rule, since it is both a product and a raw material.


If AI does trade, it probably has to be a set of rules of its own. Like AI can easily sell all net food, once it has 1k*planets. Very smart version would probably even understand that selling food is better than building energy districts in early game. But this is probably an overkill.

I see, well then I understand your confusion because that is a false assumption. The AI will not only do focus goals until they are all fulfilled, it will just weight those resources higher when scoring. For example if i have an alloy income of 30 and a mineral income of 35 then my focus goal for alloys will not be fulfilled, therefore its score will get multiplied by a defined factor. My mineral focus goal is fulfilled and will not get that extra multiplier, HOWEVER since the relative lack in income for minerals ( 200 - 35 = 165 ) is very high that will affect its scoring by a lot and will in the end probably end up being higher than the alloy score even though it has a focus bonus, since the relative income diff for alloys is much lower ( 100 - 30 = 70 ). The base income goals are NEVER disregarded.

Well the alloys would be spent on things such as ships, ship upgrades, ship upkeep, starbases, starbase upgrades, starbase modules and finally as a trade resources. Having a somewhat sizable alloy stockpile is also important when going into the midgame if you get into wars and need to rebuild your fleet at a moments notice. It's a very important resource all in all. Also there is nothing stopping the AI from building more outposts or ships than they have cap for so those numbers don't tell the whole story I think.

And I'm not sure if I'm reading you correct but by default the early game is about 100 years so then 50 alloys should be pretty fine as an early focus goal, no?

I'm sorry if you interpreted my comment as sass or any kind of implication that you do not know the game, that was not my intention! ( my bad )

The goal is to have the AI play the game using the same rules as the player, because that makes it much easier to understand and plan for as a human player. Therefore such solutions as having special jobs disabling systems for the AI or just having them cheat really isn't preferred and we have gotten a lot of criticism for doing that in the past ( even the scaling resource bonuses for difficulty ). Those kinds of solutions are also often not generic enough, I would rather try to design a system that is generic and robust enough to not need to do these kinds or special solutions because even if it's much more difficult it is also much more maintainable and easy to improve upon when the game is patched/updated in the future.

Also I just straight up disagree that consumer goods and alloys don't need a focus goal, which is also backed up by the tests that we performed when developing this. Without an alloy focus goal the AI could not stockpile enough alloys to react to their fleets being destroyed, also having an alloy income too low slowed their expansion significantly. Consumer goods are also way to important to fall behind on because it's one of the resources used most by important buildings as upkeep, if you cannot build those kinds of buildings at any time or have to fight a deficit then that is much more difficult and hindering than having too much that you can just sell on the market. Especially since production of consumer goods and alloys require a building slot, which you might not always have ( building districts is much easier in that regard ). Being proactive in this regard showed much better results than being reactive.

 

[sidestep]

FYI, my quiz question wasn't hypothetical. I regularly saw this happen in Stellaris before I modded in a fix. So I can pretty much guarantee this will keep occuring if you didn't take this into account in your AI rework.



In that case I hope you'll keep in mind the idea to set the desired surplus in the strategic resources file rather than by a central plan.



I hope you can confirm if I'm understanding things correctly.

Am I understanding correctly that if the surplus goals of the plan are met, the AI will switch back to looking at the current system of weights?

Does the AI completely ignore weights until that point?

For instance suppose the AI is currently producing 100 physics research and 100 society research, while the goal is to reach 200 of both. Suppose I have two research buildings, one that provides physics research, the other society. Suppose there's a planet that has a +40% society research modifier, and I have given a weight to the society research building to make it more likely to be built on planets with that research modifier. Will the AI actually look at that weight, or will it simply look for any research building to meet its central plan, meaning there's a 50% chance for it to build the physics building or society building?

Oh well, then I'm sure that QA knows of it too, they seem to know most things, haha.

I will definitely keep it in mind ^^

Yes, if you fulfill your plan, i.e. all income goals are met, then the AI will use building weights again. As for your example, the new system takes into account the 40% bonus as well, because it will look at where a building will produce the most before building it and that will affect the score. However, if it has the choice between building a building that produces 8 society or 4 of each research then it will probably build the one that produces 4 of each just because it relatively fulfills more of the overall goals and also gets a small bonus from the planet modifier. Is that clarification enough?

 

[sidestep]

These changes look great!
Looking forward to the patch!
Can the AI change plans dynamically based on events? (e.g. a crisis spawns or war is declared and the AI switches to a plan with higher alloy production?) Would something like this be scriptable?

Is there a has_ai_plan condition?

You could script this in a plan, you could give a high alloy plan a much higher weight if there is a crisis around for example.
There is no has_ai_plan condition I think, but that should be easy to add

 

[SeekingEtermity]

Then you probably have done some calculations. Default settings early game is 1200 months, say average would be 30. 36k alloys where would they theoretically go?
Forget 100 years, let's talk first 20. 240*30=7200 alloys. What are they going to be spent on?

Hmm, first 20 years of a not-very-warlike, not-rapidly-expanding game...

• 12 outposts (1200 alloy)
• 3 of those get upgraded (600 alloy)
• Fill the modules (50 each x2 per starbase) and building (average 75 each) slots on the upgraded starbases (525 alloy)
• 4 science ships (400 alloy)
• 3 colony ships (600 alloy)

I haven't built or upgraded a single military vessel, nor have I built any defense platforms, and I've already used 3325 alloy, nearly half of the total. Filling out your initial corvette fleet (probably about 110/ship, depends on your techs and fittings) is 1870 alloys if you only need 17 ships, or 2970 if you have Distinguished Admiralty. That leaves you with 2005 alloys and one small fleet, or 905 and a medium one. What can you do with those?

• Build some (most, if your fleet size is 20) of another fleet?
• Buy a few more science ships so you can have a decent roster of scientists gaining experience through survey, anomalies, digs, or assisting research, and ready to jump in at need?
• Build some defense platforms to harden a choke point?
• Replace losses from battles with spaceborne aliens and/or an early war?
• Expand to another 5 systems because you're xenophobe and influence doesn't cap your early expansion so badly?

Pick, like, 1-3 of those. That's not much. It's not bad - I think if you're not facing an early war (from either side) then it's a decent amount of alloys to target - but it's not going to cut it if your neighbor is an advanced start genocidal or there's a 1000 fleet strength space amoeba flock occupying a system you really want or whatever. Still, I think for the first 20 years, averaging 30 alloys a month is better than most players can or need to manage, sure.

You start talking the first HUNDRED years, though? At the end of the first hundred years, I'm rolling fleets of size 80 or more, probably at least two of them. My ships have also definitely gotten more expensive, at least 150 alloys/size point and probably more like 200. At 150 each, 36000 alloys provides 240 size points of ship, which is an entirely reasonable amount when trying to take over the galaxy and inevitably taking losses in war. It doesn't leave so much as a single alloy for outposts, starbase upgrades, modules, starbase buildings, civilian ships of any stripe, or platforms. Depending on your game settings, you'll probably have built around 40 outposts (4000 alloys), upgraded at least six of them at least twice (3600 alloys), built the modules and buildings this provided (2100 alloys), colonized at least 6 worlds (1200 alloys), and built at least 6 science ships and probably an other constructor (700 alloys). All up, that's 11600 alloys, about a third of the total. Your fleets (including losses and upgrades), defense platforms, and so on will have to come out of the rest, and let me tell you, against a competent enemy (a player, or even a Starnet AI on normal difficulty) that won't be enough.

Indeed, I think one of the great flaws of this system is that the by-default-100-year-long game phases are just too long for this kind of plan. There's a reason the Soviets set 5-year plans, not 100-year plans. Having a 30 alloy surplus 20 years into the game is "All right, you're showing real effort" territory. Having 50 alloys surplus at 100 years into the game is "... are you even awake in there?" territory. You should have at least twice that much and ideally more like 8 times that much (not too hard; each Foundry Arcology on an ecumenopolis is going to be outputting roughly 50 alloys after bonuses, and a single non-ecu Foundry World might be putting out 180 or so with the tech and population you can have by mid-game). Even on 0.25x habitable worlds, 200 alloys/month surplus would be a better goal for the lead-in to 2300.

 

[UltimateTobi]

@sidestep @MatRopert Another question came to mind: the AI is excluded from a lot of anomalies and has been excluded from some mechanics (back then from Edicts for example).

Have you enabled the AI to use all available mechanics (including anomalies)?

If not, can you name exceptions?

Thanks again!

 

[Verx90]

Hmm, first 20 years of a not-very-warlike, not-rapidly-expanding game...

• 12 outposts (1200 alloy)
• 3 of those get upgraded (600 alloy)
• Fill the modules (50 each x2 per starbase) and building (average 75 each) slots on the upgraded starbases (525 alloy)
• 4 science ships (400 alloy)
• 3 colony ships (600 alloy)

I haven't built or upgraded a single military vessel, nor have I built any defense platforms, and I've already used 3325 alloy, nearly half of the total. Filling out your initial corvette fleet (probably about 110/ship, depends on your techs and fittings) is 1870 alloys if you only need 17 ships, or 2970 if you have Distinguished Admiralty. That leaves you with 2005 alloys and one small fleet, or 905 and a medium one. What can you do with those?

• Build some (most, if your fleet size is 20) of another fleet?
• Buy a few more science ships so you can have a decent roster of scientists gaining experience through survey, anomalies, digs, or assisting research, and ready to jump in at need?
• Build some defense platforms to harden a choke point?
• Replace losses from battles with spaceborne aliens and/or an early war?
• Expand to another 5 systems because you're xenophobe and influence doesn't cap your early expansion so badly?

Pick, like, 1-3 of those. That's not much. It's not bad - I think if you're not facing an early war (from either side) then it's a decent amount of alloys to target - but it's not going to cut it if your neighbor is an advanced start genocidal or there's a 1000 fleet strength space amoeba flock occupying a system you really want or whatever. Still, I think for the first 20 years, averaging 30 alloys a month is better than most players can or need to manage, sure.

You start talking the first HUNDRED years, though? At the end of the first hundred years, I'm rolling fleets of size 80 or more, probably at least two of them. My ships have also definitely gotten more expensive, at least 150 alloys/size point and probably more like 200. At 150 each, 36000 alloys provides 240 size points of ship, which is an entirely reasonable amount when trying to take over the galaxy and inevitably taking losses in war. It doesn't leave so much as a single alloy for outposts, starbase upgrades, modules, starbase buildings, civilian ships of any stripe, or platforms. Depending on your game settings, you'll probably have built around 40 outposts (4000 alloys), upgraded at least six of them at least twice (3600 alloys), built the modules and buildings this provided (2100 alloys), colonized at least 6 worlds (1200 alloys), and built at least 6 science ships and probably an other constructor (700 alloys). All up, that's 11600 alloys, about a third of the total. Your fleets (including losses and upgrades), defense platforms, and so on will have to come out of the rest, and let me tell you, against a competent enemy (a player, or even a Starnet AI on normal difficulty) that won't be enough.

Indeed, I think one of the great flaws of this system is that the by-default-100-year-long game phases are just too long for this kind of plan. There's a reason the Soviets set 5-year plans, not 100-year plans. Having a 30 alloy surplus 20 years into the game is "All right, you're showing real effort" territory. Having 50 alloys surplus at 100 years into the game is "... are you even awake in there?" territory. You should have at least twice that much and ideally more like 8 times that much (not too hard; each Foundry Arcology on an ecumenopolis is going to be outputting roughly 50 alloys after bonuses, and a single non-ecu Foundry World might be putting out 180 or so with the tech and population you can have by mid-game). Even on 0.25x habitable worlds, 200 alloys/month surplus would be a better goal for the lead-in to 2300.

I think you are exagerating . But this does made me make another question .

How does.those plans change with changes to the game rules ? Apart from difficulty and scaling , how about science and unity cost ?

In the case of games with faster tech and unity , 100 years may be indeed a half the game .