Stellaris Dev Diary #172 - Reworking the AI

[FinellyTrained]

I see, well then I understand your confusion because that is a false assumption. The AI will not only do focus goals until they are all fulfilled, it will just weight those resources higher when scoring. For example if i have an alloy income of 30 and a mineral income of 35 then my focus goal for alloys will not be fulfilled, therefore its score will get multiplied by a defined factor. My mineral focus goal is fulfilled and will not get that extra multiplier, HOWEVER since the relative lack in income for minerals ( 200 - 35 = 165 ) is very high that will affect its scoring by a lot and will in the end probably end up being higher than the alloy score even though it has a focus bonus, since the relative income diff for alloys is much lower ( 100 - 30 = 70 ). The base income goals are NEVER disregarded.

So, we are talking net, right?

i have an alloy income of 30 and a mineral income of 35

AI_DEFICIT_SCORE_MULT = 50                    # AI will score buildings producing resources in deficit this much more
        AI_FOCUS_SCORE_MULT = 10                    # AI will score buildings producing focus resources this much more
        AI_AMENITIES_SCORE_MULT = 2                    # AI will score amenity buildings this much more than other misc resources
        AI_HOUSING_SCORE_MULT = 5                    # AI will score housing buildings this much more than other misc resources
        AI_CRIME_REDUCTION_SCORE_MULT = 2            # AI will score crime fighting buildings this much more than other misc resource
        AI_ADMIN_CAP_SCORE_MULT = 2                    # AI will score admin cap buildings this much more than other misc resource
        AI_POPS_SCORE_MULT = 5                        # AI will score pop growth and assembly buildings this much more ( already fairly high weighted  in code )
        AI_UPGRADE_SCORE_MULT = 40                    # AI will score building upgrades this much more ( since they don't take up a new building slot )

minerals_deficit = 200-35 = 165
alloy_deficit = 100-30 = 70

Foundry weight = 70 * AI_DEFICIT_SCORE_MULT * AI_FOCUS_SCORE_MULT = 70 * 50 * 10 = 35000
Mining district weight = 165 * AI_DEFICIT_SCORE_MULT = 165 * 50 = 8250
Foundry wins.

What did I get wrong?

Well the alloys would be spent on things such as ships, ship upgrades, ship upkeep, starbases, starbase upgrades, starbase modules and finally as a trade resources. Having a somewhat sizable alloy stockpile is also important when going into the midgame if you get into wars and need to rebuild your fleet at a moments notice. It's a very important resource all in all. Also there is nothing stopping the AI from building more outposts or ships than they have cap for so those numbers don't tell the whole story I think.

To tell the whole story with numbers we will have to go very deep into the numbers. My point is that expenses I listed are the most sources to sink alloys into. Upgrades, not so much. Upkeep is irrelevant, we talking about net figures. The idea of having a stockpile of alloys big enough to rebuild the whole fleet (or double it) is sound. But it has to be calculated.

And I'm not sure if I'm reading you correct but by default the early game is about 100 years so then 50 alloys should be pretty fine as an early focus goal, no?

My point is mostly that early game is long. 50-100 alloys is ok by the end of it, but I am not sure if it is the best way to set it as an early goal. Tests though are good enough arbiter of this. If this system does well, than even being generous and assuming that I am right, does not necessarily mean that the system is bad enough to warrant a rework.

I'm sorry if you interpreted my comment as sass or any kind of implication that you do not know the game, that was not my intention! ( my bad )

Well, it was an attempt to make you smile. I guess it failed. That is my bad. Sorry about that.

The goal is to have the AI play the game using the same rules as the player, because that makes it much easier to understand and plan for as a human player. Therefore such solutions as having special jobs disabling systems for the AI or just having them cheat really isn't preferred and we have gotten a lot of criticism for doing that in the past ( even the scaling resource bonuses for difficulty ). Those kinds of solutions are also often not generic enough, I would rather try to design a system that is generic and robust enough to not need to do these kinds or special solutions because even if it's much more difficult it is also much more maintainable and easy to improve upon when the game is patched/updated in the future.

I am 100% behind you on this one. Though I am not sure, that increased incomes from stations are the best way to go about it. Buffing empire default income or adding couple of levels to AI leaders or giving it discounts on ships might work too.

Also I just straight up disagree that consumer goods and alloys don't need a focus goal, which is also backed up by the tests that we performed when developing this. Without an alloy focus goal the AI could not stockpile enough alloys to react to their fleets being destroyed, also having an alloy income too low slowed their expansion significantly. Consumer goods are also way to important to fall behind on because it's one of the resources used most by important buildings as upkeep, if you cannot build those kinds of buildings at any time or have to fight a deficit then that is much more difficult and hindering than having too much that you can just sell on the market. Especially since production of consumer goods and alloys require a building slot, which you might not always have ( building districts is much easier in that regard ). Being proactive in this regard showed much better results than being reactive.

That idea was based on my misunderstanding, so should be disregarded. I will look it over again some time later and maybe even strike it out.

 

[sidestep]

Hmm, first 20 years of a not-very-warlike, not-rapidly-expanding game...

• 12 outposts (1200 alloy)
• 3 of those get upgraded (600 alloy)
• Fill the modules (50 each x2 per starbase) and building (average 75 each) slots on the upgraded starbases (525 alloy)
• 4 science ships (400 alloy)
• 3 colony ships (600 alloy)

I haven't built or upgraded a single military vessel, nor have I built any defense platforms, and I've already used 3325 alloy, nearly half of the total. Filling out your initial corvette fleet (probably about 110/ship, depends on your techs and fittings) is 1870 alloys if you only need 17 ships, or 2970 if you have Distinguished Admiralty. That leaves you with 2005 alloys and one small fleet, or 905 and a medium one. What can you do with those?

• Build some (most, if your fleet size is 20) of another fleet?
• Buy a few more science ships so you can have a decent roster of scientists gaining experience through survey, anomalies, digs, or assisting research, and ready to jump in at need?
• Build some defense platforms to harden a choke point?
• Replace losses from battles with spaceborne aliens and/or an early war?
• Expand to another 5 systems because you're xenophobe and influence doesn't cap your early expansion so badly?

Pick, like, 1-3 of those. That's not much. It's not bad - I think if you're not facing an early war (from either side) then it's a decent amount of alloys to target - but it's not going to cut it if your neighbor is an advanced start genocidal or there's a 1000 fleet strength space amoeba flock occupying a system you really want or whatever. Still, I think for the first 20 years, averaging 30 alloys a month is better than most players can or need to manage, sure.

You start talking the first HUNDRED years, though? At the end of the first hundred years, I'm rolling fleets of size 80 or more, probably at least two of them. My ships have also definitely gotten more expensive, at least 150 alloys/size point and probably more like 200. At 150 each, 36000 alloys provides 240 size points of ship, which is an entirely reasonable amount when trying to take over the galaxy and inevitably taking losses in war. It doesn't leave so much as a single alloy for outposts, starbase upgrades, modules, starbase buildings, civilian ships of any stripe, or platforms. Depending on your game settings, you'll probably have built around 40 outposts (4000 alloys), upgraded at least six of them at least twice (3600 alloys), built the modules and buildings this provided (2100 alloys), colonized at least 6 worlds (1200 alloys), and built at least 6 science ships and probably an other constructor (700 alloys). All up, that's 11600 alloys, about a third of the total. Your fleets (including losses and upgrades), defense platforms, and so on will have to come out of the rest, and let me tell you, against a competent enemy (a player, or even a Starnet AI on normal difficulty) that won't be enough.

Indeed, I think one of the great flaws of this system is that the by-default-100-year-long game phases are just too long for this kind of plan. There's a reason the Soviets set 5-year plans, not 100-year plans. Having a 30 alloy surplus 20 years into the game is "All right, you're showing real effort" territory. Having 50 alloys surplus at 100 years into the game is "... are you even awake in there?" territory. You should have at least twice that much and ideally more like 8 times that much (not too hard; each Foundry Arcology on an ecumenopolis is going to be outputting roughly 50 alloys after bonuses, and a single non-ecu Foundry World might be putting out 180 or so with the tech and population you can have by mid-game). Even on 0.25x habitable worlds, 200 alloys/month surplus would be a better goal for the lead-in to 2300.

A lot of interesting points and I do agree with you that producing a lot of alloys really is THAT important, but just a clarification, the early game alloy goal is 100 ( 50 is just the focus ) and also there is nothing stopping the AI from going higher if it can

 

[sidestep]

I think you are exagerating . But this does made me make another question .

How does.those plans change with changes to the game rules ? Apart from difficulty and scaling , how about science and unity cost ?

In the case of games with faster tech and unity , 100 years may be indeed a half the game .

The early/mid/late game targets change with game length in the game settings, so this should be fine. I do think I will change it so that it can take mid/late game plans if they finish the early game plan before the early game is over, if I get around to it ( time and priorty, time and priority )

 

[SeekingEtermity]

early_default_plan = { type = early income = { energy = 50 minerals = 200 food = 50 consumer_goods = 50 alloys = 100 unity = 50 physics_research = 200 society_research = 200 engineering_research = 200 exotic_gases = 1 volatile_motes = 1 rare_crystals = 1 sr_living_metal = 1 sr_zro = 1 sr_dark_matter = 1 } focus = { energy = 10 minerals = 30 alloys = 50 food = 10 consumer_goods = 20 } pops = 500 empire_size = 1.25 potential = { country_uses_consumer_goods = yes country_uses_food = yes } ai_weight = { weight = 1 } }

A few questions about this:

1. What is the AI supposed to do in between getting the 100 alloys/month (2270, assuming no bonus from difficulty) and the 1 each of Living Metal, Zro, and Dark Matter (quite easy to not even have access to all of these in the first 100 years)? How does it set priorities after it's fulfilled everything it can but there's a bunch of stuff it can't?
2. Why in the world would you try to get to 50 CG/month surplus? CGs are just about the quintessential "you produce these only because they're required for upkeep" resource; they have exactly one way to directly spend them (other than selling) and it's rarely used, almost never critical, and not that expensive. Yes, your usage can spike when you conquer somebody's research world or something, but that's not a major risk this early in the game.
3. A bunch of these are wayyyyy too low for targets heading into 2300. Like, by then I usually have started at least two ecus (and hopefully more), but at 200/month that's 25 years of doing nothing at all that spends minerals aside from enacting those two decisions. 50 energy is similarly horrifyingly insufficient, especially if you want to terraform much.
4. Can you use a years_elapsed filter to break these goals down with more granularity? Like, 5-year-plans would be perhaps a bit much, but perhaps 20-year-plans?
5. Do these scale at all with the number of habitable worlds? I routinely blow well past a lot of these counts before 2300 except the pops, if I'm playing a genocidal on a 0.25x habitable worlds galaxy. Sure, hopefully it'll be possible to play with more worlds after the perf fixes, but it turns out 0.25x feels like the right amount and 1x feels like WAY too many (plus the micro gets to be a huge hassle). Worlds should each matter, dammit! And yet, I can still exceed these numbers easily, without even any AI bonus to help me snowball.

Don't get me wrong, I LOVE what I'm seeing in this dev diary. However, it really does feel like these budget plans are too broad and these targets aren't great.

One thing I'd love to see is the ability to have triggered modifiers to these numbers, or at least have them target variables. Bump up the targets over time. Prioritize (or deprioritize) alloys based on how many rivals and treaties you have. Use counts or variables or flags to make the food surplus target scale with the number-of-colonies count.

EDIT: I misread the alloys/month target.

 

[Balesir]

2. Planetary plans would be really nice, and the planet automation and planed designations could be tied more into the economic AI, but as always: can't do everything

Actually, it seems to me that this AI would be really simple to adapt into a good potential Sector AI. Maybe you could allow the player to assign a sector to some Plan(TM) for control, maybe with settings for the actual resource levels used for targets? Then the Sector would run as a "mini empire" trying to reach those (player set/influenced) goals?

 

[GuildenSpur]

I just wanted to chime in again and say that I am really impressed at how @sidestep and @MatRopert have answered way more questions and more elaborate than probably ever before in a Stellaris Dev Diary.
Thank you!

 

[sidestep]

So, we are talking net, right?

AI_DEFICIT_SCORE_MULT = 50                    # AI will score buildings producing resources in deficit this much more
        AI_FOCUS_SCORE_MULT = 10                    # AI will score buildings producing focus resources this much more
        AI_AMENITIES_SCORE_MULT = 2                    # AI will score amenity buildings this much more than other misc resources
        AI_HOUSING_SCORE_MULT = 5                    # AI will score housing buildings this much more than other misc resources
        AI_CRIME_REDUCTION_SCORE_MULT = 2            # AI will score crime fighting buildings this much more than other misc resource
        AI_ADMIN_CAP_SCORE_MULT = 2                    # AI will score admin cap buildings this much more than other misc resource
        AI_POPS_SCORE_MULT = 5                        # AI will score pop growth and assembly buildings this much more ( already fairly high weighted  in code )
        AI_UPGRADE_SCORE_MULT = 40                    # AI will score building upgrades this much more ( since they don't take up a new building slot )

minerals_deficit = 200-35 = 165
alloy_deficit = 100-30 = 70

Foundry weight = 70 * AI_DEFICIT_SCORE_MULT * AI_FOCUS_SCORE_MULT = 70 * 50 * 10 = 35000
Mining district weight = 165 * AI_DEFICIT_SCORE_MULT = 165 * 50 = 8250
Foundry wins.

What did I get wrong?



To tell the whole story with numbers we will have to go very deep into the numbers. My point is that expenses I listed are the most sources to sink alloys into. Upgrades, not so much. Upkeep is irrelevant, we talking about net figures. The idea of having a stockpile of alloys big enough to rebuild the whole fleet (or double it) is sound. But it has to be calculated.



My point is mostly that early game is long. 50-100 alloys is ok by the end of it, but I am not sure if it is the best way to set it as an early goal. Tests though are good enough arbiter of this. If this system does well, than even being generous and assuming that I am right, does not necessarily mean that the system is bad enough to warrant a rework.



Well, it was an attempt to make you smile. I guess it failed. That is my bad. Sorry about that.



I am 100% behind you on this one. Though I am not sure, that increased incomes from stations are the best way to go about it. Buffing empire default income or adding couple of levels to AI leaders or giving it discounts on ships might work too.



That idea was based on my misunderstanding, so should be disregarded. I will look it over again some time later and maybe even strike it out.

Net income yes, it's all net income. First of all you got the calculations wrong, there is a lot more that goes into it than that, its not just a straight up multiplier of the income diff. The choice of buildings are also a cutoff weighted random so even if something has a higher score there is a chance that something else is built.

Upkeep is not irrelevant since it can change at a moments notice when you bring bigger fleets out of orbit ( where it costs a lot less upkeep to have them sit around ). In the end what we saw during tests and are reflected in how the AI performs is that it is better to have a too big stockpile and income of alloys than having to little, WAY better. At least from this AIs perspective.

Haha, no, dangit. I got it wrong. I'm smiling now that I got it though so, it worked out in the end

Yeah, I there are a lot of things we could do to improve the AI in different ways. I wish we had the time and resources to, I really do. But we must also be careful how we go about doing these changes and find out where we can get the most improvement for the least "cost".

It's fine, it's easy for me to talk about and so on since I designed the system and know exactly how it works. Its harder for you as a third party reader of a DD.

 

[Methone]

the early game alloy goal is 100 ( 50 is just the focus )

Jesus Christ! Are we planning for AI to be fighting an entire galaxy of federated Determined Exterminators? Or is this just a generous 'safety margin'?

 

[sidestep]

A few questions about this:

1. What is the AI supposed to do in between getting the 100 alloys/month (2270, assuming no bonus from difficulty) and the 1 each of Living Metal, Zro, and Dark Matter (quite easy to not even have access to all of these in the first 100 years)? How does it set priorities after it's fulfilled everything it can but there's a bunch of stuff it can't?
2. Why in the world would you try to get to 50 CG/month surplus? CGs are just about the quintessential "you produce these only because they're required for upkeep" resource; they have exactly one way to directly spend them (other than selling) and it's rarely used, almost never critical, and not that expensive. Yes, your usage can spike when you conquer somebody's research world or something, but that's not a major risk this early in the game.
3. A bunch of these are wayyyyy too low for targets heading into 2300. Like, by then I usually have started at least two ecus (and hopefully more), but at 200/month that's 25 years of doing nothing at all that spends minerals aside from enacting those two decisions. 50 energy is similarly horrifyingly insufficient, especially if you want to terraform much.
4. Can you use a years_elapsed filter to break these goals down with more granularity? Like, 5-year-plans would be perhaps a bit much, but perhaps 20-year-plans?
5. Do these scale at all with the number of habitable worlds? I routinely blow well past a lot of these counts before 2300 except the pops, if I'm playing a genocidal on a 0.25x habitable worlds galaxy. Sure, hopefully it'll be possible to play with more worlds after the perf fixes, but it turns out 0.25x feels like the right amount and 1x feels like WAY too many (plus the micro gets to be a huge hassle). Worlds should each matter, dammit! And yet, I can still exceed these numbers easily, without even any AI bonus to help me snowball.

Don't get me wrong, I LOVE what I'm seeing in this dev diary. However, it really does feel like these budget plans are too broad and these targets aren't great.

One thing I'd love to see is the ability to have triggered modifiers to these numbers, or at least have them target variables. Bump up the targets over time. Prioritize (or deprioritize) alloys based on how many rivals and treaties you have. Use counts or variables or flags to make the food surplus target scale with the number-of-colonies count.

EDIT: I misread the alloys/month target.

1. So this is an interesting issue. The plan has no idea if you actually can fulfill it or not, the "unattainable" resources are there so that if you actually CAN get them you will try harder to, BUT a plan can still be considered fulfilled if you get "close enough" to fulfill it. So if you for example have all the resource goals ecept ZRO then it will be considered fulfilled. What we have seen is that most AI don't 100% fulfill the early game plan on normal difficulty so it works more like a set of guidelines than hard goals. If they fulfill it they will continue building things still so that's not really an issue.
2. As I said before, the CG goals is mostly about keeping a buffer and giving it a weight to rival the other resources in the scoring. If the AI builds a lot of CG buildings early on it can just switch over to building science labs or sell them and be fine, if they get into a CG deficit they have a much harder time.
3. You are way better than our Normal difficulty AI then, also these goals were never meant to match players. They were goals set by QA to reflect a Normal difficulty AI and give them something to strive for, they can still end up producing more
4. You could just have multiple early game plans broken down by years_elapsed yeah, for sure. But they would have to be separate plans unfortunately.
5. They don't I'm afraid, they kind of assume a "normal" setup, I wish I had time to make them more dynamic and "better" but this turned out to be a good approximation ^^

 

[Spaceception]

So there was a comment here about fleet builds, but I can't find it, and it doesn't look like its been answered

Will the AI properly fit its fleets with a good amount of weapons and defenses, and more importantly, will it counter, or even hard counter the player/other AIs, and (at least) the mid/late-game crisis?

I've seen AI with PD despite no missiles on either side, disruptors on ships that are just wasted slots because you need a lot to have an impact, and even empty slots altogether later on (like the X slot), have these been fixed?

 

[FinellyTrained]

Net income yes, it's all net income. First of all you got the calculations wrong, there is a lot more that goes into it than that, its not just a straight up multiplier of the income diff. The choice of buildings are also a cutoff weighted random so even if something has a higher score there is a chance that something else is built.

Yes, I know that they are wrong, hence the question. I would really like to understand the algorithm for this. If you could spare of your time to show a correct calculation or know a link to something that describes it or what Stellaris folder to look into that would be great.

Upkeep is not irrelevant since it can change at a moments notice when you bring bigger fleets out of orbit ( where it costs a lot less upkeep to have them sit around ). In the end what we saw during tests and are reflected in how the AI performs is that it is better to have a too big stockpile and income of alloys than having to little, WAY better. At least from this AIs perspective.

Oh, yeah, that. AI should calculate and use the real fleet maintenance in calculations. So parked fleet will generate some bonus, which should be ignored for balancing.

Yeah, I there are a lot of things we could do to improve the AI in different ways. I wish we had the time and resources to, I really do. But we must also be careful how we go about doing these changes and find out where we can get the most improvement for the least "cost".

100% behind you on this too. But a lengthy discussion might allow for more cost-benefit weighted ideas to implement.

It's fine, it's easy for me to talk about and so on since I designed the system and know exactly how it works. Its harder for you as a third party reader of a DD.

Well, I do not know should we blame it on your lack of explanation skills or my lack of comprehension skills (smile here), but to give you meaningful feedback, I need to understand how it works.

 

[sidestep]

Actually, it seems to me that this AI would be really simple to adapt into a good potential Sector AI. Maybe you could allow the player to assign a sector to some Plan(TM) for control, maybe with settings for the actual resource levels used for targets? Then the Sector would run as a "mini empire" trying to reach those (player set/influenced) goals?

This is a really interesting idea, and on paper it looks like a really good idea. I do think however that adapting this to the Sector AI in practice is much more difficult. But I'll keep it in mind

 

[HugsAndSnuggles]

Jesus Christ! Are we planning for AI to be fighting an entire galaxy of federated Determined Exterminators? Or is this just a generous 'safety margin'?

Considering that some players fight (and kill) x5 crisis by the time this goal is supposed to be reached, it's pretty lenient.

 

[sidestep]

Jesus Christ! Are we planning for AI to be fighting an entire galaxy of federated Determined Exterminators? Or is this just a generous 'safety margin'?

 

[Methone]

Considering that some players fight (and kill) x5 crisis by the time this goal is supposed to be reached, it's pretty lenient.

So we're basing the AI off of the players who zerg-rush enemy homeworlds with a science ship and their fleet right behind it and get 1 million power by 2300?

Because that seems silly.

 

[HugsAndSnuggles]

So we're basing the AI off of the players who zerg-rush enemy homeworlds with a science ship and their fleet right behind it and get 1 million power by 2300?

Because that seems silly.

No, you can, actually, manage that with relatively well-paced, peaceful tech-focused playthrough, AFAIK (not necessarily technocracy, either). Might need better-than-average conditions to pull it off in some cases, but still.

 

[Hertzila]

So we're basing the AI off of the players who zerg-rush enemy homeworlds with a science ship and their fleet right behind it and get 1 million power by 2300?

Because that seems silly.

I think it would be better described as giving the AI at least a snowball's chance in hell to delay the people that do this or the 5x crisis, so they'd serve as some sort of a speed bump in case of foreign aggression.
In other words...

Jesus Christ! Are we planning for AI to be fighting an entire galaxy of federated Determined Exterminators?

Yes.

 

[SeekingEtermity]

empty slots altogether later on (like the X slot), have these been fixed?

Oh man, few bugs in Stellaris make me so mad as seeing that the enemy finally got around to building battleships, and even has a spinal mount weapon option, and then just didn't use it and consequently left the ship with 2/3 the firepower it could have had with the normal artillery bow and maybe only about 50%-60% as much as if it had actually filled the X slot.

 

[sidestep]

Yes, I know that they are wrong, hence the question. I would really like to understand the algorithm for this. If you could spare of your time to show a correct calculation or know a link to something that describes it or what Stellaris folder to look into that would be great.



Oh, yeah, that. AI should calculate and use the real fleet maintenance in calculations. So parked fleet will generate some bonus, which should be ignored for balancing.



100% behind you on this too. But a lengthy discussion might allow for more cost-benefit weighted ideas to implement.



Well, I do not know should we blame it on your lack of explanation skills or my lack of comprehension skills (smile here), but to give you meaningful feedback, I need to understand how it works.

Yeah, no, I get it. But I won't be sharing any code examples in a DD since these things change all the time and would make little sense out of context of the code. But simplified you can look at it along these lines:
Starting from a base weight, you look at your current income, goal income and what the building in question actually produces. For each resource produced that is also part of the goal you add that to the weight of the building scaled ( with some modification ) by how much we are currently missing to reach that goal. Effectively meaning that if you have a building that produces a lot of something that we need a lot of, that would add a lot of weight. This then becomes the new "base weight" that we scale with all the different focus/deficit factors.

Once again, this is a simplification and more stuff goes into it but this is the basic idea. I wanted to get away from the "if this then add this much weight" because it was just not fine grained or dynamic enough.

Yeah I agree that discussions and feedback from you guys ( the community ) is really valuable, which is why I'm here talking to you all
In the end though there is just so much more that goes into the realities of game development that even if you have the greatest idea or most amazing discussion on the forums there just isn't anything that can come of it, that's just how it is. Hopefully this is not one of those cases and we can actually get around to do some of the improvements you've all come up with ( or some of the improvements we devs have all come up with already, which many of you have mirrored ) in the end.

 

[Balesir]

This is a really interesting idea, and on paper it looks like a really good idea. I do think however that adapting this to the Sector AI in practice is much more difficult. But I'll keep it in mind

I'm sure the devil will be in the details, because it invariably is, but at least it might give a "starter for 10" for the Sector AI rework.