• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Stellaris Dev Diary #385 - AI Benchmarks

Hi everyone!

The 4.0.13 update released today with the following changes:

Stellaris 4.0.13 Patch​

Improvements​

  • Behemoth Fury is now available to Wilderness Empires.
  • Improved tooltips for the following civics:
    • Functional Architecture/Constructobot
    • Environmentalist
    • Astro-mining Drones
    • Maintenance Protocols
    • Ascensionists
    • Augmentation Bazaars
    • Brand Loyalty
    • Death Cults
    • Dimensional Worship

Balance​

  • Mutagenic Habitability now counts all planet types as ideal for upgrading Gaiaseeders
  • Dramatically increased the draw chance for the Mineral Purification, Global Energy Management and Food Processing technologies
  • Rebalanced the Pleasure Seeker civics to transform Civilians into Hedonists
  • Logistic Drones are now Complex Drones not Menial Drones

Bugfix​

  • Fixed invaded pre-ftls not becoming biotrophies
  • People once more die when they are put in the Lathe
  • Bio-Swarmer missiles can now be used by all biological ships with medium weapon slots (including defensive platforms)
  • Pops that are being pampered will now be forcibly switched to the correct living standard
  • Replacing a district specialisation no longer destroys CyberCreed buildings that should be kept
  • Corrected a tooltip bug where a planet would display itself as a possible migration target.
  • Fixed capitalisation for resources in trade policies
  • Updated assorted modifiers that still referred to Clerks
  • A Trade deficit now causes Job Efficiency and Empire Size issues
  • Fixed the tooltip for the Polymelic trait
  • Armies now protect 200 pops from raiding, not 2
  • Blocked the Federation Code technology for some empires, for example homicidals. To draw the tech, the empire is also required to be in contact with someone they can form a federation with.
  • Blocked the Development focus task Form a Federation for some empires, for example homicidals
  • Added swaps for some empires, for example homicidals, for the Development focus rewards Federation Code, Xeno Diplomacy, and Xeno Relations
  • Updated the Colony view tab mentioned in the hint of the focus task Enact a Planetary Decision to say Management
  • CyberCreed pops with Ritualistic Implants can now colonise planets
  • Fixed Recycled and Luxurious traits not applying to Roboticists
  • Catalytic Processing Civic now lists correct information regarding job swap
  • Cost for repairing orbital rings when you use bioships is now correctly calculated
  • Gale Speed trait gained from Defeat no longer causes errors
  • Fixed scope for LeaderShipSurvivalReason
  • Fixed scope bug for ruler in leader_election_weight
  • Fixed Worker Coop gaining Elite strata jobs in too many places and tidied up the civic tooltip
  • Updated tooltip for Warrior Culture civic
  • Added a pre-list colon to the Feudal Society civic's tooltip
  • The everychanging stone can no longer cause artisans to have negative mineral upkeep
  • Gave the Neural Chorus advanced authority the pop growth speed modifier that had accidentally been assigned to Memory Aggregator
  • The Planetary Supercomputer no longer has an empire cap of 1
  • The Research Institute/Planetary Supercomputer no longer give scientist capacity
  • Added dashes to Traits tooltips and list items
  • Fixed trigger logic for criminal syndicates and federations
  • Fixed Offspring Bioships not being visible in game
  • Fixed Offspring Bioships not being labelled as non-offspring ships in the ship designer
  • The Machine Uprising will no longer spawn 100 machine pops for every 1 missing housing. However the pop-rework seems to have handled 6 million machine pops okay.
  • Stopped removing occupation armies for bombarded and invaded planets on savegame load
  • Repairing ruined buildings in zones is now always possible.

Performance​

  • Flattened pop job modifier node into planet one
  • Made clearing modifiers a fire and forget job

Stability​

  • Fixed a possible OOS when a player leaves the game.
  • Fix CTD when generating a Cosmic Storm mesh.
  • Fixed a random freeze when loading save with stations containing multiple defence platforms.

We expect the 4.0.14 release will be next week (probably on Tuesday), and is expected to include some fixes to a few infinite loops and some select balance changes (like splitting up Enforcers and Telepaths again). It will be a short work week here in Sweden, so it’s likely to be the only update of the week.

As I mentioned last week, with multiplayer stability largely handled, AI is one of our next focuses. Today I want to talk about AI benchmarks, and have a discussion with you about how we should measure “success”.

What Makes a Good AI?​

The AI in Stellaris has always been designed as very reactive, and AI personality has a massive impact on their behavior. Our goal is for our AI empires to feel like actors in the galactic play - acting in a manner consistent with their Origins, Authorities, Civics, and Ethics rather than always picking the “meta” play.

They do still need to put up a bit of a challenge though, especially at higher difficulties.

The first economic goal we make for our AI is “please don’t collapse in an economic death spiral”, and it’s actually far better at that in 4.0 than it was in 3.x. The current AI does NOT meet the second “provide an adequate challenge” goal though.

One of the fundamental tools we have for our AIs are resource targets in their economic plans. They’ll strive to reach those targets, and many of these are set as “scaling” - if they meet the target, they’ll raise the target the next month. This attempts to ensure that they’ll keep thirsting for ever larger research and alloy numbers (or food if they use bioships!) as is appropriate. This is one of the tools we also use to make them exhibit their ethics - Materialists scale their Research targets faster than other empires, so they’ll inherently be more likely to build more Research specializations, while Spiritualists are more likely to have a lot of Unity specializations.

Ironically, improving AI tends to consume any benefits we carved out through performance improvements. The stronger the AI, the more stuff they have - fleets, colonies, and so on.

Benchmarking​

One way to decide whether or not the AI is performing up to expectations is through benchmarking - what kind of fleet power, alloy generation, and research generation should they have by 2230, 2250, 2300, and so on? Around what year should they hit 10k fleet power?

Then there come questions around whether the benchmarks should differ based on personality type. Should it be different if they’re Democratic Crusaders vs. Peaceful Traders? Or does differentiating them there make the friendlier empires too weak?

I’ve got my own set of benchmarks that come from running 3.14 and from the multiplayer community, and in general, I’m okay with Grand Admiral being significantly harder than it was in 3.14. but I’m interested to hear what you all strive for.

How much research and alloy production do you try to have 10 years, 30 years, 100 years, and when the end-game crisis comes calling? (Include your preferred difficulty settings and galaxy sizes as well if you could, as well as if you change any other important settings like tech costs.)

What’s Next?​

We’re going to continue with 4.0 post release support.

Since the next two weeks are both short weeks in Sweden, our next Stellaris Dev Diary will be June 12th. (You’ll be hearing from me in patch notes in the meantime though.)
 
  • 76Like
  • 16
  • 5
Reactions:
I have no clue what any of this means but I am Content
 
  • 6Haha
  • 1
Reactions:
Enjoy your short week!

I’m not too sure on the numbers, it varies enough between players and empire types that I’ll have to have a think on it. EDIT: Potentially nonviable idea but what if the benchmarks were dynamic to the player? On ensign the AI would attempt to meet 80% of the player's metrics, whereas on grand admiral it might be 800%.

For AI in general I do have a couple of comments:

1) Planet Management: Probably the most notable issue with the AI in terms of the economy is how poorly developed their planets are compared to a player. They will do things like:

  • Designate planets in ways that don’t match their resources (e.g make a high mineral world an energy world)
  • Build buildings in random locations, like having a lab in their urban slots rather than a research enclave slot. This takes up space in the former that could go to another building
  • Not redevelop their worlds so their home worlds stay on generic mixes with a hodgepodge of buildings, or even just keep buildings like luxury residence around when they don’t need them

These are all noticeable when taking over an AI world leading to a lot of management to delete things and redevelop.

2) AI diplomacy: I love the goal of making AI work as characters in a narrative sense. For me this immersion is often broken due to a few behaviours:

  • Continually supporting random resolutions like security contractors, even if there’s no benefit
  • Spamming the same federation law requests only to want to revert them a few years later (like migration on or off)
  • Wanting to become a vassal even if they have powerful defensive pacts, guaranteed independence, friends in a federation etc. This one is particularly egregious as it’s poor to have an ally you’ve been friends with for decades randomly offer to give up their sovereignty when they don’t need to. And it interacts very badly with federations, since the AI will often refuse to join a federation but will happily become a vassal even if it pulls them into a fed with no say. IMO the AI should, with some exceptions for personality, consider vassalisation as their last diplomatic option when there are powerful threats.

Please consider these, particularly how the AI calculates vassal willingness in the presence of other factors (treaties, federations etc). Even a quick fix of the AI using the same logic of "won't join a federation with X" when the requesting overlord is in a federation with X would be good (without taking into account any federation policies since they could change).
 
Last edited:
  • 56Like
  • 16
  • 7
Reactions:
Hi everyone!

The 4.0.13 update released today with the following changes:

Stellaris 4.0.13 Patch​

Improvements​

  • Behemoth Fury is now available to Wilderness Empires.
  • Improved tooltips for the following civics:
    • Functional Architecture/Constructobot
    • Environmentalist
    • Astro-mining Drones
    • Maintenance Protocols
    • Ascensionists
    • Augmentation Bazaars
    • Brand Loyalty
    • Death Cults
    • Dimensional Worship

Balance​

  • Mutagenic Habitability now counts all planet types as ideal for upgrading Gaiaseeders
  • Dramatically increased the draw chance for the Mineral Purification, Global Energy Management and Food Processing technologies
  • Rebalanced the Pleasure Seeker civics to transform Civilians into Hedonists
  • Logistic Drones are now Complex Drones not Menial Drones

Bugfix​

  • Fixed invaded pre-ftls not becoming biotrophies
  • People once more die when they are put in the Lathe
  • Bio-Swarmer missiles can now be used by all biological ships with medium weapon slots (including defensive platforms)
  • Pops that are being pampered will now be forcibly switched to the correct living standard
  • Replacing a district specialisation no longer destroys CyberCreed buildings that should be kept
  • Corrected a tooltip bug where a planet would display itself as a possible migration target.
  • Fixed capitalisation for resources in trade policies
  • Updated assorted modifiers that still referred to Clerks
  • A Trade deficit now causes Job Efficiency and Empire Size issues
  • Fixed the tooltip for the Polymelic trait
  • Armies now protect 200 pops from raiding, not 2
  • Blocked the Federation Code technology for some empires, for example homicidals. To draw the tech, the empire is also required to be in contact with someone they can form a federation with.
  • Blocked the Development focus task Form a Federation for some empires, for example homicidals
  • Added swaps for some empires, for example homicidals, for the Development focus rewards Federation Code, Xeno Diplomacy, and Xeno Relations
  • Updated the Colony view tab mentioned in the hint of the focus task Enact a Planetary Decision to say Management
  • CyberCreed pops with Ritualistic Implants can now colonise planets
  • Fixed Recycled and Luxurious traits not applying to Roboticists
  • Catalytic Processing Civic now lists correct information regarding job swap
  • Cost for repairing orbital rings when you use bioships is now correctly calculated
  • Gale Speed trait gained from Defeat no longer causes errors
  • Fixed scope for LeaderShipSurvivalReason
  • Fixed scope bug for ruler in leader_election_weight
  • Fixed Worker Coop gaining Elite strata jobs in too many places and tidied up the civic tooltip
  • Updated tooltip for Warrior Culture civic
  • Added a pre-list colon to the Feudal Society civic's tooltip
  • The everychanging stone can no longer cause artisans to have negative mineral upkeep
  • Gave the Neural Chorus advanced authority the pop growth speed modifier that had accidentally been assigned to Memory Aggregator
  • The Planetary Supercomputer no longer has an empire cap of 1
  • The Research Institute/Planetary Supercomputer no longer give scientist capacity
  • Added dashes to Traits tooltips and list items
  • Fixed trigger logic for criminal syndicates and federations
  • Fixed Offspring Bioships not being visible in game
  • Fixed Offspring Bioships not being labelled as non-offspring ships in the ship designer
  • The Machine Uprising will no longer spawn 100 machine pops for every 1 missing housing. However the pop-rework seems to have handled 6 million machine pops okay.
  • Stopped removing occupation armies for bombarded and invaded planets on savegame load
  • Repairing ruined buildings in zones is now always possible.

Performance​

  • Flattened pop job modifier node into planet one
  • Made clearing modifiers a fire and forget job

Stability​

  • Fixed a possible OOS when a player leaves the game.
  • Fix CTD when generating a Cosmic Storm mesh.
  • Fixed a random freeze when loading save with stations containing multiple defence platforms.

We expect the 4.0.14 release will be next week (probably on Tuesday), and is expected to include some fixes to a few infinite loops and some select balance changes (like splitting up Enforcers and Telepaths again). It will be a short work week here in Sweden, so it’s likely to be the only update of the week.

As I mentioned last week, with multiplayer stability largely handled, AI is one of our next focuses. Today I want to talk about AI benchmarks, and have a discussion with you about how we should measure “success”.

What Makes a Good AI?​

The AI in Stellaris has always been designed as very reactive, and AI personality has a massive impact on their behavior. Our goal is for our AI empires to feel like actors in the galactic play - acting in a manner consistent with their Origins, Authorities, Civics, and Ethics rather than always picking the “meta” play.

They do still need to put up a bit of a challenge though, especially at higher difficulties.

The first economic goal we make for our AI is “please don’t collapse in an economic death spiral”, and it’s actually far better at that in 4.0 than it was in 3.x. The current AI does NOT meet the second “provide an adequate challenge” goal though.

One of the fundamental tools we have for our AIs are resource targets in their economic plans. They’ll strive to reach those targets, and many of these are set as “scaling” - if they meet the target, they’ll raise the target the next month. This attempts to ensure that they’ll keep thirsting for ever larger research and alloy numbers (or food if they use bioships!) as is appropriate. This is one of the tools we also use to make them exhibit their ethics - Materialists scale their Research targets faster than other empires, so they’ll inherently be more likely to build more Research specializations, while Spiritualists are more likely to have a lot of Unity specializations.

Ironically, improving AI tends to consume any benefits we carved out through performance improvements. The stronger the AI, the more stuff they have - fleets, colonies, and so on.

Benchmarking​

One way to decide whether or not the AI is performing up to expectations is through benchmarking - what kind of fleet power, alloy generation, and research generation should they have by 2230, 2250, 2300, and so on? Around what year should they hit 10k fleet power?

Then there come questions around whether the benchmarks should differ based on personality type. Should it be different if they’re Democratic Crusaders vs. Peaceful Traders? Or does differentiating them there make the friendlier empires too weak?

I’ve got my own set of benchmarks that come from running 3.14 and from the multiplayer community, and in general, I’m okay with Grand Admiral being significantly harder than it was in 3.14. but I’m interested to hear what you all strive for.

How much research and alloy production do you try to have 10 years, 30 years, 100 years, and when the end-game crisis comes calling? (Include your preferred difficulty settings and galaxy sizes as well if you could, as well as if you change any other important settings like tech costs.)

What’s Next?​

We’re going to continue with 4.0 post release support.

Since the next two weeks are both short weeks in Sweden, our next Stellaris Dev Diary will be June 12th. (You’ll be hearing from me in patch notes in the meantime though.)
How to evaluate AI? Let's take the Grand Admiral difficulty with an increase towards the middle of the game. This difficulty should be at the level of a good player at year 50, not a noob. At year 100, the same. A player at such difficulty settings, on average, does not want to feel that by year 100 the AI is behind in economics, science and navy.

Playing out personalities is, of course, good. But make the AI a worthy competitor, not a dummy and an extra. If you can't create an illusion of intelligence, throw even more buffs at it.
 
  • 11
  • 6
  • 1Like
  • 1Haha
Reactions:
In an optimal world, the AI would not have to cheat to be powerful (ex. Grand Admiral = free resources for AI), but instead it would make better strategic choices, which would result in better fleets / economies overall.

lower difficulties is fine for the AI to do less optimal decisions, but anything higher would have to mean that the AI is actually competent and not what it is today. A competent player 10/10 times will win against an AI opponent even with maximum difficulty, because the AI is not inherently that intelligent ast it stands and has been an issue for the wider Stellaris community regarding their actual performance and decision making.

What I am hoping for is that the AI will be able to provide an actual challenge, which can for example be statistically measured to be better via benchmarking and I look forward to seeing the solutions brought fourth by the Devs to address the issue of AI competency.

Thank you Paradox for finally addressing the AI issue.
 
  • 9Like
  • 3
  • 1
  • 1
Reactions:
“please don’t collapse in an economic death spiral” - in 4.0.X, I still often see AI building a colony ship, but then refusing to actually use it.

Like I met The Chosen, depopulated a few of their planets including the homeword (but not the other 2 planets in the homeworld system), and they would never recolonize their homeworld, despite having a colony ship in the system for decades.

Later I wiped them out, and to this day, nobody wants to colonize the now empty cluster, but I have seen science ships scanning them, so they know how to use a wormhole. I wonder why AI does not see a system with 3 gaia worlds as something worth colonizing.
 
  • 19Like
  • 5
  • 2
Reactions:
mayber give logistic drone upkeep and higher output too

really low right now

also it is very difficult to tell what is increasing the job proficiency of a job

please make better tooltip
 
  • 2Like
  • 1
Reactions:
Bioship empires now can't get access to the Explosive weapon repeatables, since they have tech_missile_3 as requirement - which bioships cannot have.
That should be fixed, especially since even the Behemoth uses a lot of swarmer missiles.
 
  • 11
  • 2Like
  • 1
Reactions:
When do we get fixes for gestalt machine intelligences with machine worlds AP / resource consolidation origin? Machine worlds "rural" districts (the bottom 3) still don't change description text to what they are specialized in and don't provide any housing at all. In addition, the "Malfunctioning replicator bay" on the resource consolidation origin homeworld doesn't add any assembly jobs. If this is intended please remove the entire blocker, or rename it, or make it make early game pop numbers go "brrrrrrrrrr" again :3

Plz <3
 
  • 5Like
  • 1
Reactions:
> I’m interested to hear what you all strive for.
> How much research and alloy production do you try to have 10 years, 30 years, 100 years, and when the end-game crisis comes calling? (Include your preferred difficulty settings and galaxy sizes as well if you could, as well as if you change any other important settings like tech costs.)

Can't you gather the data directly from the game app? Wouldn't it provide a much better sample than asking people on the forum?
A genuine question, no pressure.
 
  • 14Like
Reactions:
@Eladrin @PDS_Iggy
Can we please get energy, mining and food support districts for Wilderness empires?
They are shown in promo video, but for unknown reason they're scrapped and not in the final game. This would immensely help us specialize our planets properly.
1747913138351.png
 
  • 7
  • 4Like
  • 1Love
Reactions:
have noticed the ai is back to making loads of habitats and now citadels(and not very good placed ones either) any plans on adding an ai limit for these again?
 
  • 3Like
  • 1
Reactions:
I’d like the AI to be more interactive with other AI players.

In most of my games the AI just spends most of the time declaring every other faction a rival then doing nothing about it.

Or they federate & sit there doing nothing.

And then a mid or late game crisis happens & im lucky if one AI faction has the competence and/or resources to deal with it if it happens in their space.

Also, when AI is at war, why do they seem to prioritise troop transports & sconce ships?
 
  • 5
Reactions:
Thank you for that honest update concerning the AI of the game. However some statements really bother me.

1. The fact that you know that the AI is no challenge in the moment and was better in 3.14 draws me to the conclusion that it was never in focus during the development of the dlc . If this is so I am a little puzzled about your priorities since stellaris is mostly played in SP…

2. taken the fact that the AI was better in 3.14 and my system performance in 4.0.X is actually worse and taking your statement that an improvement of the AI in 4.0.X would lead to performance losses again draws me to the conclusion that the new pop system has only a minor impact on the game performance . Worst case would be that the performance improvements you saw in your tests simply came from the fact that the AI cannot manage the new system …

3. asking the community how many ressources an empire should have at a certain time on a certain difficulty draws me to the conclusion that your internal QA mostly relies on console commands to trigger certain gameplay situations and not on actual playing a full game. And I am sorry to say it: you asking us how to best fix the AI sounds a little desperate. I am not a game developer…

Do not get me wrong I love you guys for a game I always enjoyed . But this time I feel more than ever like a beta tester and I fear that we run in a cities skylines 2 or prison architect 2 situation here . A project to big to really fix in a reasonable time…
 
Last edited:
  • 38Like
  • 18
  • 9
  • 3
Reactions:
but instead it would make better strategic choices, which would result in better fleets / economies overall.
The easiest way to make that tbh is just to have the AI play meta build, but if you do that, you'd end up with some really hideous strategies.
 
  • 1Like
  • 1
Reactions:
One way to decide whether or not the AI is performing up to expectations is through benchmarking - what kind of fleet power, alloy generation, and research generation should they have by 2230, 2250, 2300, and so on? Around what year should they hit 10k fleet power?

The aim should be just to make the AI competent enough at handling its deficits and its forward planning so that it doesn't completely stagnate, then any excess resources will gradually fill up into the other important stuff. It's pretty much completely dependent on galaxy size / habitable density at this point.

Right now the player can reach like 12K research per month by midgame with civil education / psionic ascension with very little effort, which by itself throws balance estimations out the window. Prior to 4.0 a 'good mid-game income' would include atleast 100-250 alloys a month with a good one or two forge worlds, and maybe 1-2K research if you had enough planets. Hardcore players could get way past 1k alloys and 3K+ research even prior to that. Research is just as long as you could afford it, spam tech worlds.

On 3.14, grand admiral hits 10k fleet power maybe 20 years into the game. On ensign, 50-60. If it's stable. In 4.0, the majority of the AI's strength is Federation fleets, Mercenary fleets, and the occasional summoned dimensional fleet.

One issue that exists currently is that the AI doesn't know how or when to demolish buildings correctly (such as Hydroponics Farms, Luxury Residence), as the scripts that exist for that currently will check against its entire income, before deciding to demolish every instance of that building. Do not ever use potential = {} for this logic. Yes it is rare, but it still isn't good.

Another major pitfall that i don't believe was fixed; the AI had tended to over-prioritise Strategic Resources - for a good reason - but it was at the expense of the rest of its economy. Reducing this in testing my slightly smarter AI mod, i discovered that the AI *will not understand* that it can't upgrade buildings if it doesn't have enough motes/etc, which became far more apparent in a country that didn't have ANY income of that resource. This would completely brick its planner, as it would be trying to upgrade a mining hub (during mineral deficit!!!) but not have the motes to do so. In 4.0 refineries now apply a base income to metallurgists/artisans - which i think is too far in the other direction.

The AI at least knows how to build Zones, but with the high base cost and the AI's seeming inability to replace them, they become very restrictive for future planning. Planets will end up with ANY zone, usually science or unity, but then the AI won't build any industry during a consumer goods deficit because it doesn't remove its existing Zones.

Lastly, crime. The AI doesn't know how to deal with crime. They just don't. Without being able to demolish buildings in its own planning, worlds can end up with 100% crime and never be dealt with, or have several now useless precincts that will stay there til the rest of the game.

God i write too much. Anyway it's that.
 
  • 16Like
  • 8
  • 6
Reactions: