Performance Megathread

[GuildenSpur]

Are you intensionally trying g to take offense to everything?
On medium games you get more maurauders, more FE, more leviathans etc.
That's not being on a high horse, that's how Stellaris sets up games. Again I do not care how other people play their games. Everybody enjoys the game how he or she wants.

 

[Mauer]

So because you have to experience everything in a single run in order for it to be the "True Stellaris Experience", doing it any other way is the wrong way to play? Are you intentionally mounting that high horse?

I play small and sometimes tiny for performance and I wish I could select more FEs and get more unique systems like the bigger galaxies do; it's not about experiencing everything every game but sometimes you get nothing unique, specially if playing small/tall. I barely ever get to see any leviathans anymore.

 

[Chaines]

Agreed, i only play small galaxy with fewer planets because endgame lag drives me crazy, and i would certainly play big galaxy if i could. That's what i did at first and i got much more stories to tell, but never finished a game due to bad performances.

 

[exi123]

For me the game gets much more life when i can play a huge galaxy with many empires, marauders and Fallen Empires. I dont need many habitable planets tho, building megastructures was and is a cool way to get the lategame interesting when the borders are getting harder and the "natural" living space is used.

Looking forward to Federations it is actually needed to get this better. Whats the purpose of federations when you only have 4 empires left when the lategame kicks in.

 

[alexti]

I'm thinking about returning to playing Stellaris (last played about year ago) but even then the lag was killing my enjoyment of the game... What is your experiance with lag on medium or small galaxies with few habitable planets? My specs: amd ryzen 5 2600, geforce 1060 6GB and 16 GB of RAM. I don't want to waste few hours of my free time just to learn that after 2300 I will be unable to play damn game...

Playing on ~6 years old box on medium or small with default number of planets I don't see much lag, I can stay under 10-15 sec/month though I have to turn down the graphics. Though the playstyle might play significant role, as it was discussed in this thread.

 

[alexti]

I don't see why Stellaris is all about huge maps, bigger isn't always better. I play Small/Medium galaxies (with a higher number of AI's than standard), with a bunch of mods perfectly fine.

To me it's the micromanagement more than the lag that stops me from playing on large/huge map. Unlike many other PDS games, Stellaris does fairly good job of minimizing the tedium (by allowing players to choose smaller maps), so I totally agree that it plays better on small/medium.

 

[James Fire]

Are you intensionally trying g to take offense to everything?
On medium games you get more maurauders, more FE, more leviathans etc.
That's not being on a high horse, that's how Stellaris sets up games. Again I do not care how other people play their games. Everybody enjoys the game how he or she wants.

The way you said it initially heavily implied that playing on small galaxies was the wrong way to play.

 

[James Fire]

I play small and sometimes tiny for performance and I wish I could select more FEs and get more unique systems like the bigger galaxies do; it's not about experiencing everything every game but sometimes you get nothing unique, specially if playing small/tall. I barely ever get to see any leviathans anymore.

Less AI empires then, turn up the primitives. Primitives don't have an exclusion zone around them that blocks hostiles from spawning.

 

[Mauer]

Less AI empires then, turn up the primitives. Primitives don't have an exclusion zone around them that blocks hostiles from spawning.

Interesting. Do you know if said exclusion zones can be modded out?

 

[James Fire]

Interesting. Do you know if said exclusion zones can be modded out?

Probably.

Or at the least, you can force spawn systems with hostile entities near, or next to, home systems after the game starts.

 

[Uburian]

TLDR: There’s no tldr; I spent weeks experimenting trying to understand what’s going on and finding a solution including many hours to write this up. Now that the title grabbed your attention, please read this post to understand fully what’s going on.


4. The POPs are innocent: How it actually works (approximately!).

Everyone, including me, stated that the problem was the number of pops and the fact that the game spends too much time processing them. That is wrong. The true problem is vacant jobs. When a planet has no vacant jobs, no processing is taking place, as in:

“When a planet has no vacant jobs, the game engine doesn’t touch a single POP datum. It doesn’t even read them from ram. It’s zero processing”


The pops don’t check for jobs. it’s the other way around: the jobs check for pops and we all know how this goes:

a. Every month:

b. For each vacant job:

c. Check if even employed pops can fill it – check all pops.

So, if you have a colony with even a single job and 300 pops, that’s 300 checks, including:

i. Pulling all pop data including all traits.

ii. Pulling population rights, and calculating if a pop can be eligible for promotion.

iii. Calculating and comparing weights for both jobs: the job you are looking for and the job the current employed pop candidate you are inspecting is already doing.

Performance is O(c*v*p), c: number of colonies, v: number of vacant jobs, p: number of pops

Now, I really wish this was optimized so I don’t know if the game groups jobs and pops for performance because then the performance model changes. The resettlement window shows pops grouped. I’m not sure if this is just a visual thing or if it has engine significance. Also not sure what, if any, caching is done and if it can be relevant at these large data set numbers – more on that later.

Unless they optimize pop computation to increase bandwith to mannage the data set, it's pointless, becasue then you also need to keep checking on planets that are full of pops - I'd always prefer jobs checking for pops because then planets with no jobs are skipped (which is the common case?)

But regardless, you can always double your pops and double the workload so:

There are 2 correct solutions, beyond remodeling the system, in my books: One is to create a reactive system that responds to events and does no daily ongoing calculation. Everyone would accept the burden of massive processing if a huge change would happen in your empire in effort for the engine to reach a new state, but not the ongoing constant extreme lag.

The other correct solution is to have a system where changes happen slowly across the planets with minimal lag, but in that case edicts, regulations and other game rules might take years to take effect. You would see inconsistencies on pop alocations and other things - that's real bureaucracy for you! This second system would never lag, but as empires get larger you might need 20+ in-game years for policy changes to take effect. You might remember playing such a system in simcity 4 when developing zones.

Excellent job GnoSIS! A very insightful read.

I concur that your proposed second solution would be the best way to both optimize the game dramatically and make it, at least for my taste, way more realistic. However, i'm fully aware that such a change would require a very extensive rework of the job system, and taking that in consideration i think that your first proposal is optimal, for a time at least.

Simply put, and if i have understood your analysis correctly, the game wastes an extreme amount of processing capacity in job checks that are not necessary at all.

The current job check system:

1. Almost constant (Monthly checks).
   
2. Universal (Checks for all vacant jobs and compares them to all pops galaxy wide, no matter if they are unemployed or not, planet per planet).
3. Excludes from the calculations only planets that have no vacant jobs. (Which is very counter-intuitive for both the player).
4. The result is a system that works well enough in the early game, but scales really poorly as the game progresses, coming to an almost complete halt in the late mid or early late game (at least under the conditions that the game is supposed to be played) and penalizes the player for planning ahead (building districts and buildings before the pops needed to fill them are created).

Damn, that is inefficient and messy. If i had to make a guess, i would say that they created this check system before reworking the pop growth and migration systems, under the assumption that late game planets with vacant jobs would the exception rather than the rule. However, in late mid/early late game the filling of jobs is either at the hand of an almost useless migration system (that needs a full rework on its own) or a very micromanagement heavy resettlement mechanic that is as optimized as the job check system itself. Result: The player colonizes/captures a planet/habitat/ringworld, queues the necessary districts and buildings for said planet, resettles some pops to it (only if it is an option) and forgets about it. The Ai more or less works under the same (logical and intuitive) principles, and both them and the player/s dramatically decrease the performance of the game as a consequence as the game goes on, just for doing the logical thing.

How can we solve this mess?

1. The job check system needs a complete rework. as you pointed out, a more dynamic system would be optimal and lag free, but for the time being a more reactive system would be enough.
2. The migration and resettlement systems need a complete rework, not only to optimize the game performance but to make the game more enjoyable by reducing micromanagement.

How can we make the job check system more reactive and less performance intensive?

1. Eliminate the monthly universal check.
2. Check for a specific planet/habitat/ringworld ONCE when a district/building is constructed/repaired.
3. Check for a specific planet/habitat/ringworld ONCE when a pop is created, migrates or is resettled.
   
4. Check for a specific planet/habitat/ringworld ONCE when the player/Ai sifts the job priorities or planet focus.
   
5. Check for a specific planet/habitat/ringworld ONCE when said planet has one of its species genetically or robotically modified.
   
6. Check for a specific planet/habitat/ringworld ONCE when said planet ownership changes to another empire.
   
7. Check for a specific empire ONCE when said empire changes its ethics/civics/policies/traditions/ascension perks.

WIth this system, the game checks only when it needs to, and most planets don't consume processing power most of the time except when they are actively growing/changing. Unused job spaces are not calculated unless there is a new pop that could fill them, or if a substantial change happens in its host planet/empire. This system is far from perfect (it would still be susceptible to slow downs caused by late game planetary development, war, and empire evolution), but at the very least it would be more reactive, and less performance heavy than what we have now.

These are just some ideas i had about the topic though. I'm not familiar enough with the game's code to know if changes like these could work, but i wanted to share them nevertheless.

Uburian

 

[GnoSIS]

Excellent job GnoSIS! A very insightful read.

I concur that your proposed second solution would be the best way to both optimize the game dramatically and make it, at least for my taste, way more realistic. However, i'm fully aware that such a change would require a very extensive rework of the job system, and taking that in consideration i think that your first proposal is optimal, for a time at least.

Simply put, and if i have understood your analysis correctly, the game wastes an extreme amount of processing capacity in job checks that are not necessary at all.

The current job check system:

1. Almost constant (Monthly checks).
   
2. Universal (Checks for all vacant jobs and compares them to all pops galaxy wide, no matter if they are unemployed or not, planet per planet).
3. Excludes from the calculations only planets that have no vacant jobs. (Which is very counter-intuitive for both the player).
4. The result is a system that works well enough in the early game, but scales really poorly as the game progresses, coming to an almost complete halt in the late mid or early late game (at least under the conditions that the game is supposed to be played) and penalizes the player for planning ahead (building districts and buildings before the pops needed to fill them are created).

Damn, that is inefficient and messy. If i had to make a guess, i would say that they created this check system before reworking the pop growth and migration systems, under the assumption that late game planets with vacant jobs would the exception rather than the rule. However, in late mid/early late game the filling of jobs is either at the hand of an almost useless migration system (that needs a full rework on its own) or a very micromanagement heavy resettlement mechanic that is as optimized as the job check system itself. Result: The player colonizes/captures a planet/habitat/ringworld, queues the necessary districts and buildings for said planet, resettles some pops to it (only if it is an option) and forgets about it. The Ai more or less works under the same (logical and intuitive) principles, and both them and the player/s dramatically decrease the performance of the game as a consequence as the game goes on, just for doing the logical thing.

How can we solve this mess?

1. The job check system needs a complete rework. as you pointed out, a more dynamic system would be optimal and lag free, but for the time being a more reactive system would be enough.
2. The migration and resettlement systems need a complete rework, not only to optimize the game performance but to make the game more enjoyable by reducing micromanagement.

How can we make the job check system more reactive and less performance intensive?

1. Eliminate the monthly universal check.
2. Check for a specific planet/habitat/ringworld ONCE when a district/building is constructed/repaired.
3. Check for a specific planet/habitat/ringworld ONCE when a pop is created, migrates or is resettled.
   
4. Check for a specific planet/habitat/ringworld ONCE when the player/Ai sifts the job priorities or planet focus.
   
5. Check for a specific planet/habitat/ringworld ONCE when said planet has one of its species genetically or robotically modified.
   
6. Check for a specific planet/habitat/ringworld ONCE when said planet ownership changes to another empire.
   
7. Check for a specific empire ONCE when said empire changes its ethics/civics/policies/traditions/ascension perks.

WIth this system, the game checks only when it needs to, and most planets don't consume processing power most of the time except when they are actively growing/changing. Unused job spaces are not calculated unless there is a new pop that could fill them, or if a substantial change happens in its host planet/empire. This system is far from perfect (it would still be susceptible to slow downs caused by late game planetary development, war, and empire evolution), but at the very least it would be more reactive, and less performance heavy than what we have now.

These are just some ideas i had about the topic though. I'm not familiar enough with the game's code to know if changes like these could work, but i wanted to share them nevertheless.

Uburian

Thank you for the reply and comment.

No one is familiar with the game code, only the people that have access to it. Some of us can see a bit deeper because of our profession, experience and hard work. Don't forget that this looks to be just a single source of lag. In the Federation of Planets save, some of the empires had dozens of sub species in their population which compounds the issue. Trade and migrationn just add on top and if all were optimized something else would come up. It's the nature of things.

A fix is needed for sure, but only a fix that would make this game playable up to a specific benchmark say 10k pops galaxy wide (a silly nnumber). I'm willing to accept a global pop cap around which the game would be playable and supported by paradox as the games' limit - pops wouldn't grow beyond that and this would be interesting as a different game: kill pops to grow your own. Sure, let the user override the cap in the set up game options or with the console, but then he should loose the right to complain about it. This was done on other PDX games where support is availiable only if you're playing within a specific date range and I love and accept that because its reasonable.

As I said regardless of any publisher effort, I can just play a bit more and double my poplulation. It's impossible for any developer and hardware to keep up with such dataset growth. But the publisher must draw that line! I imagine the majority wouldn't complain if they could play up until resolving the endgame crisis in a more or less normal capacity, no one would rightfully pay attention to the customer that uploaded his 3100+ year, 60k pop save, complaining about performance. Speaking only for myself at least, I'm not here asking for the impossible.

 

[Blayze]

This is why I've said that it's simplest to just get rid of jobs, and make everything that currently produces jobs give its resources directly.
It'd take other things being changed, like traits, unemployment and districts, sure, but then pops could be left as the expansion/development limiter they are now... and all the pop/job checks just vanish into thin air.

 

[GeneralPetrov]

Thought I'd copy-paste my post from another locked thread, because apparently civil complaints about the state of the game aren't allowed outside of mega/quarantine threads.

Just to provide some numbers to people who don't seem to have any issues with performance:

- In a full speed in a normal large galaxy at the very start of the game, one month takes approximately 6 seconds. For comparison, at full speed in a normal large galaxy around the year 2400, one month takes approximately 30 seconds, so a single day takes about 1 second. Not only this, but unlike in the early game there is rather extreme UI/input lag in the late game if you're playing on the higher speeds, so in practice the game is actually a lot slower than 30 seconds for every month.

- Again, at full speed in a normal large galaxy at the very start of the game, one year takes just over a minute. In the same amount of time at full speed in 2400, the game had only advanced a little over two months, and again this is ignoring the fact that because of the UI/input lag it is practically impossible to play at higher speeds in the late game. Basically, the game is 5 to 6 times slower in the late game than the early game at full speed.

- Taking into account the fact that the UI/input lag makes faster speeds unplayable, at normal speed where the lag is less pronounced a full month in the late game takes about 40 seconds, making the late game around 7 times slower than the early game.

- Comparisons aside, just to fully illustrate how slow the late game is, a full year in the late game around the year 2400 at normal speed will take nearly 8 minutes.

I fully understand that there will always be some slow down, obviously the late game is going to be slower to some extent than the early game. But the extent of the lag and slowdown is just completely unacceptable and unplayable. In a game that takes place across hundreds of years, a year taking 8 minutes is just ridiculous. My PC isn't a supercomputer, but it ain't no potato either, and clearly I am not the only one experiencing these performance problmes. New gameplay features, DLCs, bugfixes, and AI improvements are all worthless to me until the performance issues are resolved.

 

[lovenotwar]

Making jobs into raw numbers instead of individual pops would fix the issue really fast.
Have realistic population numbers, have districts/buildings still unlock based on population numbers, but instead of having the buildings produce based upon pops in the job, It would simply take a percentage of species population available to calculate output.
The only checks you would need is when the district or building is built to see which species would have the best output, or when you modify a species, but that would only be a check between individual species, not individual pops.
So if you had 3 different species in your empire you would have 3 checks per change, instead of 100+ per planet/ per open job / per day.
Everything would play out almost exactly the same, but you wouldn't micromanage individual jobs and you could actually make it to end game.

 

[Gort11]

I'd like there to be a point where your empire stops having individual pops, and planets become the new pops.

So when you have one planet, you assign miners and researchers and alloy foundry workers and so on. But once you have twenty planets, there's a zoom-out and you now only manage the planets - the pops are all abstracted. Instead of tracking all those individual pops, the game then only has to track "Mining planet, population 20, 60% efficient".

 

[Blayze]

I'd like there to be a point where your empire stops having individual pops, and planets become the new pops.

So when you have one planet, you assign miners and researchers and alloy foundry workers and so on. But once you have twenty planets, there's a zoom-out and you now only manage the planets - the pops are all abstracted. Instead of tracking all those individual pops, the game then only has to track "Mining planet, population 20, 60% efficient".

And then out again to "mining system," etc, I imagine.

Maybe even again to cluster.

 

[Gort11]

And then out again to "mining system," etc, I imagine.

I could certainly see this happen once you're putting multiple habitats in a system.

 

[Blayze]

I could certainly see this happen once you're putting multiple habitats in a system.

On the subject of habitats, a system-wide decision to upgrade your stations to habitats, like Mastery of Nature, would probably be best for this.

 

[LWE]

I have a lot of finished campaigns (points to AARs in sig), and the game and the AI are still in a ramshackle state. Not “unplayable”, but bad enough. Furthermore, the evidence about vacant jobs being drastically unoptimized is there, and is quite substantial. And I’m saying it as someone with few performance issues.

I’ll gladly play the game, but I won’t buy Federations if this state of things continues. Granted, I’m one of the few people who actually doesn’t think that the existing diplomacy system is that bad and who forms federations in my games sometimes.