• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Stellaris Dev Diary #239 - AI++

Доступно на русском в ВК/Read in Russian on VK

ai_3.png



Hello and welcome back to another update on the Stellaris AI. This is Guido again. Today I’m here with my fellow human Offe who also enjoys doing organic things. Like generating energy through processing photosynthesised light in the form of matter via ingestion. I like bacon and ice cream. Everybody likes bacon and ice cream. So Offe, please, take it from here.

Hello, it is me, Offe!
I’m a 28 cycles old Human manufactured and operated up here in the north. I’ve worked here at the Arctic office for two years and recently joined the Custodian team as a junior programmer. Guido and I have previously worked on other projects together and he has taught me a lot about game development, but most importantly I learned some tips on how to improve my diplomatic interaction protocols. Where I would often use phrases like “it’s an absolute disaster”, he would instead prefer “This is pretty good, but it can be even better!”. This may prove to be important later on.

I would like to say Thank You to all the people out there who took time playing on the open beta and provided us with feedback and bug reports. If you ever find the AI in a situation where it is doing something strange, please bug report and most importantly attach save games, it helps tremendously! For example, two separate issues were found and addressed with the new job changes.

And lastly, this dev diary will contain older changes and screenshots that were made long before the beta, but also new changes which were not part of the beta, meaning that you still have some new changes waiting for you in the 3.3 release.

Changes to pop job system​

I will start with this change since it will also directly affect players and not only AI!

How it used to work:

Each time something important would happen on a planet, such as a pop is grown, a district/building gets constructed or an upgrade finishes, every single pop would update their desire (also known as weight) to work each job. Then all pops would be unassigned of their jobs, and all of them would be put back on a (potentially) new job.

Now there are some pros and cons with this approach. The good thing is that we are not doing any calculations when we don’t have to, since if nothing changes then we don’t update any of the jobs. However, the downside is that if you have scripted conditional job weights, for example, based on how many amenities there are on a planet, it will cause mass migrations of pops between jobs when the system eventually does update because all pops move at the same time.

In the current 3.2 system the most obvious problem is for hive mind empires where pops will mass move to the maintenance drone job when the planet amenity level is low, and then during the next update, all of them will leave due to having way too many amenities causing a perpetual ping pong effect.

This also affected non hive mind AI empires because in 3.2 the AI would prioritize a job producing a resource during a shortage across all its planets. For example, during an energy credit shortage it would prioritize the technician job on all its planets, causing every single job to be instantly filled. This would likely cause a shortage of some other resource such as minerals, resulting in most types of AI empires to get stuck in a ping pong behaviour once they had entered a resource deficit. This also had the unfortunate side effect of AI starting constructions that were not really needed, but the sudden shift of pop jobs made it appear so.

How it works in 3.3:
- During each monthly update, update the jobs on all planets
- Only remove or add maximum of one pop per job during the update

Many of you are now probably immediately clenching your fist in anger while picturing your poor CPU melting, as scripted calculations based on number of pops in stellaris can be very CPU demanding. But I have some good news for you, first of all in 3.2 there were some redundant calls to the job weight calculation. By removing them where possible, we could already reduce the amount of job weight calculations by about 75%.

Furthermore, we are now reusing job weights between pops that are of the same species and share the same job. Meaning if you have 40 pops working as miners on a planet, and they are all of the same species, the scripted job weight calculation will only be performed once instead of 40 times as in 3.2. This comes with some limitations though, as it is no longer safe to base job weight on individual pop data, such as which faction they are in or their happiness. In the end the vast majority of all job weight calculations were removed while still updating jobs every month.

With the new system it allows you to write a scripted job weight calculation that depends on itself without causing ping pong behaviour. For example, jobs that produce amenities can now base their job weight on the planet’s amenity level, or the enforcer job can now base its job weight on the crime level.

The intention is that you will not notice any difference from the system in 3.2 other than some jobs like enforcers and maintenance drones having a more reasonable amount of pops working that job.
AYhCFhqM2Y.gif


Jobs for your pops​


In 3.2 AI would look at the number of free jobs on a planet when deciding if it needs to build new jobs. So if there were for example 3 free jobs then the AI would clap its hands together and call it a job well done and move on. At the same time the planet could have huge numbers of unemployed pops rioting on the streets.

This scenario comes from the fact that not all pops can work all jobs, so while there are technically free jobs on the planet, that doesn’t mean that the unemployed pops can actually work those jobs.

In 3.3 we are changing the way that the AI is looking at planets when it is deciding what jobs to create. Instead of looking at the number of free jobs on the planet and then creating more when this number is low, the AI will now look at actual unemployed pops and make sure to create a job that the specific pop is actually able to work.

This solves a variety of issues present in 3.2 where AI doesn’t make good decisions for pops such as slaves or robots, this is something we will continue looking at but it is a big first step in the right direction.

AI scaling economic subplans​


Scaling subplans was something we mentioned earlier as a planned feature for the future, well the future is now so strap yourself in!

In 3.2 we got rid of the old economic plans which had a predefined early/mid/late game strategy and introduced the shared base plan which doesn’t look at what year it is, but rather looks at what state the empire is in.

Now when I first saw Guido’s new economical plans I immediately thought wow this is pretty good, but it can be even better! So I started working on the scaling sub plans which aims to remove all upper limits of production (previously mentioned 500 alloy per month cap in 3.2) but still provide the AI with a responsive plan that adapts to the current state of the AI economy.

How the system works as for 3.3:
The base economic plan is now very small, it sets a minimum target for all types of strictly needed resources such as minerals, energy and food (such as +20 monthly income). Once these targets are met, then a small amount of CGs, alloys and science targets are added.

Once all of the above base plans are satisfied we then enable the scaling sub plan, which is just like any other economic plan except that it will add itself each time it is fulfilled, an unlimited amount of times. The scaling plan contains a small amount of energy/minerals but primarily contains alloys and science. This means that the more mature the AI economy becomes, the focus on base resources becomes smaller and the primary focus will shift to military and science production.

Additionally we have added 3 separate conditional scaling sub plans which we enable for materialist, militarist(and total war empires) and spiritualist empires that add additional science, alloy or unity targets to their economic plan as a first step to making AI economy more distinct from each other.

Grand Admiral hive mind reaching a monthly income of 3k alloys and 22k science in one test run by year 2422. (Screenshot from before the unity rework)

topbar.jpg


AI district/building specialization​

One of the big advantages that fellow Humans like you and I have over the AI is that we can easily make long term strategies which are based on assumptions and goals. So we may have a long term strategy to turn a planet that we have not yet colonized into a factory world. As mentioned in answers to the last AI dev diary questions, the economic AI is stateless which means that it has no notion of past nor the future, it only looks at what it has right now and what it can do to satisfy it’s economic plan. This makes it very good at adapting to the situation it is in, it will keep a close eye at the current economic situation and immediately react to any shortages but lack some of the long term planning capabilities that we have.

So how can the AI make specialized worlds without planning for the future? Well one straightforward way of doing it is simply by switching places of districts that we have already built in the past. So if we compare two planets where both of them have 5 mining and 5 energy districts each, we can gradually specialize the planets by replacing the districts one pair at a time until we end up with one planet with 10 energy districts and another with 10 mining districts.

This approach works quite well in practice and is also very dynamic in the sense that it allows the AI to make hybrid planets in the early game which becomes more specialized over time as the empire expands.

planet_view.jpg


AI consumer goods vs alloy production and planet designations​

In 3.3 we are adding an AI system where the AI will manually pick a planet designation instead of using the default scripted planet designation system which is the same one as the player gets if you do not change it yourself.

The AI system looks at the available designations for each planet and calculates how many resources it would get each month from choosing the designations. It then scores each designation by judging how well the gained resources fits into the AI’s economic plan, giving extra score to designations that align with its economic goals.

Normally it is very easy to pick the designation, for example, a planet with only mining districts on it will clearly have the mining designation. However, other designations such as Factory/Forge world are more complicated and the AI needs to carefully assign these designations in a way that keeps the economy balanced.

For non hive mind empires consumer goods and alloy production is the biggest AI economy challenge we have faced so far, since the AI needs to produce both resources independently of each other to meet their economy plan targets even though they are produced from the same district in three different possible ways. The current system is a step in the right direction but this is definitely a tricky problem that will require additional fine tuning in the future.

AI alloy spenditure​

Now that AI adjusts its alloy and consumer good production separately it was time to tackle how AI spends its alloys.

In 3.2 the AI really liked defense platforms, and keeping them up to date by upgrading them any time it was possible. Not only is this a massive drain of alloys, it would also more or less permanently fill the production queue in the shipyards with upgrades which meant that in some cases it wasn’t able to build any new ships even if it wanted to.

Further there was an issue where the AI would get blocked from building any modules or upgrading any starbases if there was an open module slot in which it wasn’t possible to build anything according to the AIs starbase templates. For example, the AI has dedicated shipyard starbase templates and if it has open slots in it then it would really like to build the titan assembly module on it. But if it wasn’t researched yet then the AI would get blocked here, preventing construction of new starbases.

In 3.3 the AI alloy spending priority goes something like this:
- Build new ships until we reach fleet cap
- Build starbase modules
- Build new starbases
- Upgrade starbases
- Upgrade ships (and defense platforms) if it gives a +30% fleet power bonus, and upgrade the entire fleet this ship is in while we are at a shipyard anyway. Saving both alloys and time!
- Build defense platforms as a last resort

AI tech picking​

The AI has scripted weights for each tech in the game, this gives it some direction as to what technology to pick next every time a research is completed. Both in terms of which technologies are more powerful but also taking into account AI personalities, militarist empires are for example more inclined to research weapon tech.

In 3.2 the majority of techs had some modifier on it which increased the chance of it being selected by the AI, but when you prioritize everything, well then you prioritize nothing. For 3.3 we went through all the techs in the game and remade the AI priorities from scratch, emphasizing techs that will help the AI scale into the mid and late game. For example, resource production boosting techs, pop growth techs and resource producing building chains are now more encouraged.

Additionally AI will now look much more favourably on techs that are cheaper compared to the other options, this allows the AI to more quickly cycle through the available options and find the techs that it really likes.



AI superfluous destruction​

This one is short and simple. AI will now delete stuff if it gives jobs, housing or building slots that we do not need. Meaning, if we for example have more free jobs and housing than provided by an energy district we will simply delete it to avoid paying the upkeep cost and freeing up this slot for something else in the future.

This scenario most often happens when an AI empire invades another planet and purges their pops, so determined exterminators will now be able to repurpose the conquered planets into something that aligns with their economy!

AI rogue servitor and bio trophies​

While there has been a lot of focus on the AI’s ability to compete economically with the player in this dev diary, one of the primary objectives of the AI initiative is also to enhance the role playing capabilities of the AI.

In 3.3 we are adding additional AI support for the rogue servitor civic and how they handle their bio trophy pops. The AI should now build an organic sanctuary on each planet that has an upgraded capital structure causing their bio trophies to spread to other planets. And they should build additional sanctuaries on planets with a lot of complex drones.

Additionally we have addressed a group of related bugs where the AI was unable to build special types of buildings like gaia seeders, spawning pools and chambers of elevation.

AI comparison​

As a final note we would like to share some comparison graphs between the 3.2 and the 3.3 AI. Please note that what you are about to see is based on one single test run on ensign and one test run on grand admiral. This comparison is not meant to be interpreted as evidence but as an indication of what has changed between 3.2 and 3.3.

In any AI playthrough there is a huge variance in the AI performance due to random factors such as how they pick techs, traditions and ascension perks. The experiment setup is also used for internal AI testing only and not representative of an actual playthrough.

Experiment setup:
  • Tiny galaxy
  • 1 AI empire
  • All test using the United Nations of Earth empire
  • Mid and late game years set to 2575/2600 so they don’t trigger
  • The map is the same between the 3.2 vs 3.3 comparison, but NOT the same between the ensign and the grand admiral test.

Let’s first look at the comparison between the 3.2 and 3.3 ensign difficulty:

image (6).png


Up until year 100 the military power is roughly the same, but from that point on the results of the work we put into mid and late game AI scaling starts to really show. This allows the AI to act and react in a lot more interesting ways in the late game than before.


image (9).png



1) Around year 150 the 3.3 (“develop”) AI reaches the 32/32 starbase capacity due to having researched all techs in the game, resulting in the slowdown of the military power development.

2) 3.2 AI gets stuck in an economic death spiral for about 30 years shortly after year 100, AI eventually manages to escape the death spiral and then has massive economic growth and is able to build up to the 32/32 starbase cap quickly due to having saved up alloys for 30~ years.

At year 200 the gap between both AI military strength gets smaller since neither AI is really building that many more ships due to having maxxed out starbase capacity and already way above their fleet cap resulting in very expensive fleets. The power gap at year 200 is mainly due to 3.3 AI having superior technology.
However, it turned out that for GA difficulty the AI wouldn’t correctly apply the increased buff from trade value. Now, when it does, the AI takes a good step in the direction of making it more challenging for players.

Overall the GA and ensign test show a similar pattern where the first 100 years are roughly the same and then the difference becomes substantial. However, in the GA test the upper limit of 3.3 AI scaling can be seen around year 150-200 as the military growth curve tends to flatten out at this point when reaching the starbase cap.

And that’s it for today's dev diary, if you have any questions related to AI economy feel free to post them below and I will do my best to answer them!
 
Last edited by a moderator:
  • 164Like
  • 63Love
  • 9
  • 3Haha
  • 3
  • 2
Reactions:
Unfortunately I can not give any good answer on that question, our primary focus has been on the normal AI empires and there are still so many areas left to improve. We have not yet discussed any plans for looking at fallen empires, but it is also sort of more in the realm of game design / content creation team and doesn't really fall purely under the AI umbrella so to speak.

To be fair, it is probably something for the custodian team to look at. The War in Heaven certainly needs a serious looking at (even eternal presidents losing their federation to the Leage of Non-Aligned Powers, the League continuing after the War in Heaven, the general lack of AI empires staying completely neutral or joining the AEs), which would also be a good chance to look at Fallen Empires.
 
  • 3Like
Reactions:
I think my logic circuits have processed your communication protocol successfully now on the second iteration.

I didn't know that you could mass allow/forbid jobs via the shift clicking stuff. I will add it on my todobedo-list.

BTW this bug also triggers when multiple pops are Assimilated -- I'm seeing multiple unemployed cyborg slaves when there are open jobs right next to them.
 
Pop updates caused when you do stuff like building buildings or demolishing stuff will still cause the pop update but it will only shift one pop at a time according to the new system. So when you build a research lab you may have 1 month where only 1/2 of the pops have filled in the specialist jobs, overall this would not matter much in the big picture but it would be possible to fix this as well for the best possible experience :)

I guess the switching is not faster b.c. the underlying Data needs to be updated first, which is too expensive to calculate multiple times a month?
My gut feeling tells me that 1 switch/month is a bit slow overall.

Question:
When the AI needs, lets say Energy, how does it choose which of the Energyjobs on all of its planets to switch first? How does it recognize that it not should fill in one job at a time on all its planet but focus on its Genertor planets instead before using the spare jobs on its capital for example?
 
AI updates are always welcome, but it worries me that these plans seem focused on the mid-lategame.

Advanced start GA die in less than 40 years, so plans kicking in around the 100 year mark is going to change absolutely nothing.
 
  • 1
Reactions:
This is more of an RP consideration as opposed to trying to absolute minmax an optimal playstyle for rogue servitors. Once the biotrophies start to reproduce they should then move back to bigger worlds with high numbers of complex drones
Doesn't seem obviously important for RP to me? So long as the servitors are breeding and caring for bio-trophies *some* places it seems to make sense for them to also have drone-only planets keeping the organic sanctuaries elsewhere supplied with food and energy.
 
A possible consideration: Make it possible for clerks to migrate, but at a much lower chance than unemployed pops (10% of base? 5%?). That way it serves like it did back when they were first added, as a sort of "buffer" job for extra pops. It'd still be unoptimal to keep pops as clerks, but only on a temporary basis, rather than a permanent one.
Could be on to something here
 
  • 11Like
  • 2
  • 1Love
  • 1
Reactions:
This all sounds great, I'm just wondering how often planetary designation calculations are performed? I'd imagine the weightings would change each time a tech that boosts a resource output is researched or pops with traits improving particular outputs arrive, but I'm concerned that might lead to the AI frequently either recalculating (which would hit performance) or redesignating (which would obviously hamper output), or alternatively sticking with one designation that becomes suboptimal over time.
I put it on the monthly update with a 1000 day cool down per planet.
 
  • 6
  • 1Like
Reactions:
While the AI and speed improvements overall do sound good and I do like the overall openness, I'm pretty alarmed to hear that you're revamping the jobs assignment logic without mention of the highly suboptimal assignments that often happen!

It's very easy to run into slaves or drone pops that are working the wrong job, and where a simple swapping of two pops' job would result in a +15% output increase.

It seems to that whatever arbitrary weight is assigned when a pop "desires" a job, especially if custom scripts are involved, shouldn't occult the actual output that will result. If worst comes to worst you could introduce a last pass after the selection of which pops will work to ensure they are placed in the slot that generates the most output: this pass would only allow swaps so wouldn't interfere with the prioritization of amenities or other needed resource. This would at least fix the scenario where pop A and B are both working jobs for which they're not particularly suited, and swapping them would at least cause one of them to benefit from some sort of bonus (+15% energy output for instance, like in the example in the thread linked above).

--

Side-note: as mentioned elsewhere I wonder if you've considered trading some extra memory for performance and pre-computing some values permanently for all existing race-job combinations. After all, once a race or job enters the game it never leaves, so the combinations created and associated values are good for the rest of the game. No point in computing them again every month. This lets you reuse the computation related to some never-changing facts (Robot race X has 15% energy production bonus) when computing values related to more dynamic elements (we're currently suffering a shortage of consumer goods).
So yes some pops work the "wrong" job in the sense that it would be possible to switch them around to minmax and get some more output, these issues are caused by the scripted weights. So the overall "rework" of the job assignment system will not make this better or worse as the problems still have to be fixed in the job weight scripts.

There was another user who posted a personal bug list with "easy to fix" bugs in the job weight scripts which seemed to be easy copy paste errors, so hopefully we can fix some of those. But again I want to reiterate that most of us developers do not sit and read these forums all day long, I am making an exception now since I wrote this DD and want to adress all the questions in here. But for the future I highly recommend posting the described issue with the combination of species/traits that cause this issue in a save game on the bug forum instead <3

About the side note, while what you say is true in principle, the scripted job weight system can be anything, such as changes in the galactic community or special buildings constructed on the planet or literally anything you can imagine. So while this would save some performance it would also limit what you would be able to do with the scripted job weights. Similar to the trade-off I made above which allows us to copy weights between pops of the same species.

Overall pop job weight calculations was a very small part of the overall game performance when I profiled it when I was making these changes. We are talking about something like a 1%~, and now it should be even less. Performance isn't something I have focused on for this patch, the primairy goal was to not make it any slower whilst still improving the experience.
 
  • 8
  • 2Like
Reactions:
I have never tried starnet myself so I can not make any statements about how 3.2+starnet vs 3.3 compares.
But you should have :( For me, this mod made is what makes the game playable in the first place. It already got AI economy and teching more than fixed, so looks like your work is cut out for you. Whatever that guy did - it just works. As much as I appreciate the notion of so-called free will and doing one's own thing, as a fellow Human you must understand the importance of submitting to superior solutions already in place, freeing the computational resource to advance the Cause in other areas.

As of now, with AI economy being fixed, the weakest point is AI military tactics, which in is neither modable nor ever mentioned in this article. This and diplomacy (who AI rivals and wars).

Also I don't see how any of this accounts for inter-state trade deals which, if played to the full of it, can relegate AI's to being mere raw recourse donors of player empire.
 
  • 2Haha
  • 1
Reactions:
have you considered making the AI "cheat" in a different way?
for example, setting the planet designation will swap 100% factory jobs to alloy and vice versa, there is no option to fine-tune it via a slider, i.e. 25% factory & 75% alloy
as a player i dont really care, besides the UI will be clunky with a slider
but the AI can have them, i dont mind, if it helps the AI make better decisions, i believe that would help make the transition into specialised factory/forge world smoother

in general, there are many aspects of the game where it will make sense to have the feature but its limited by UI, the AI can have them

another suggestion, under certain dire conditions, allow the AI access to special AI-only "IMF market" where it can make loans for resources and gain ability for instant build, they still need to pay off the loans

I sort of like this slider idea. But then it would have to be for both players and AI alike.
The easier and more obvious solution is to make the industrial district into a separate Alloy district and another Consumer Goods district.
We will never make a system for the AI which works fundamentally different from the player. The goal is to make the AI play the SAME game as the player, not a different one.
 
  • 6Like
  • 3
Reactions:
But you should have :( For me, this mod made is what makes the game playable in the first place. It already got AI economy and teching more than fixed, so looks like your work is cut out for you. Whatever that guy did - it just works. As much as I appreciate the notion of so-called free will and doing one's own thing, as a fellow Human you must understand the importance of submitting to superior solutions already in place, freeing the computational resource to advance the Cause in other areas.

As of now, with AI economy being fixed, the weakest point is AI military tactics, which in is neither modable nor ever mentioned in this article. This and diplomacy (who AI rivals and wars).

Also I don't see how any of this accounts for inter-state trade deals which, if played to the full of it, can relegate AI's to being mere raw recourse donors of player empire.
Most of the AI work has been done by fixing things in the code which mods can never change.
AI military tactics is high prio for future work!
 
  • 6Like
  • 2
  • 1Love
Reactions:
The military AI is actually pretty good, but there is an ongoing problem with order loops (which I'm sure you're aware of). Every single time I see the warfare AI performing badly, it's because it's caught in a loop.

Honestly, I think it might be worth just having some sort of blanket rule where if a fleet gets the same order three times in a short period of time it just does something to reset itself, like forces itself to a neighbouring system or something.
 
  • 2Like
  • 2
  • 1
Reactions: