3. Breakthrough.
By changing how I played, I managed to speed up late game performance 500%+. In fact, on tiny it feels like the game runs almost like day 1 of a new game/galaxy and it seems like there is no upper bound to the number of pops you can have!! You can check the attached autobots save below, load it up, see it and play it for yourself. It has about 70 colonies, and 4.5 – 5k POPs,
I wanted to scale this up and was preparing to try another test game/empire, but then AlkincTeos supplied the performance megathread in the forums with a save on the latest version of the game (with lithoids dlc activated) that was slightly modded and had 18.5k pops and 261 colonies in it. I had to give it a go and as I posted yesterday the results were equally stunning, I would never consider being able to play with 18k POPs on my hardware, let alone it actually playing butter smooth. Here is the original post with the attachments. At the end of the post, you can download and try the new saves.
https://forum.paradoxplaza.com/foru...ance-megathread.1253705/page-19#post-25965104
4. The POPs are innocent: How it actually works (approximately!).
Everyone, including me, stated that the problem was the number of pops and the fact that the game spends too much time processing them. That is wrong. The true problem is vacant jobs. When a planet has no vacant jobs, no processing is taking place, as in:
“When a planet has no vacant jobs, the game engine doesn’t touch a single POP datum. It doesn’t even read them from ram. It’s zero processing”
The pops don’t check for jobs. it’s the other way around:
the jobs check for pops and we all know how this goes:
a. Every month:
b. For each vacant job:
c. Check if even employed pops can fill it – check all pops.
So, if you have a colony with even a single job and 300 pops, that’s 300 checks, including:
i. Pulling all pop data including all traits.
ii. Pulling population rights, and calculating if a pop can be eligible for promotion.
iii. Calculating and comparing weights for both jobs: the job you are looking for and the job the current employed pop candidate you are inspecting is already doing.
Performance is O(c*v*p), c: number of colonies, v: number of vacant jobs, p: number of pops
Now, I really wish this was optimized so I don’t know if the game groups jobs and pops for performance because then the performance model changes. The resettlement window shows pops grouped. I’m not sure if this is just a visual thing or if it has engine significance. Also not sure what, if any, caching is done and if it can be relevant at these large data set numbers – more on that later.
Case study: A colony has 40 vacant jobs in 4 groups (4 different job types) and 200 existing pops working in 20 job groups:
a. No job grouping, no pop grouping: 40x200 = 8000 pair checks – Your CPU is suffering.
b. No job grouping, pop grouping: 40x20 = 800 pair checks – Still a lot!!!
c. Both pop and job grouping: 4x20 = 80 pair checks – Manageable? How about doing it for 3000 colonies?
Note: pop rights and societal strata make the actual numbers smaller – you don’t check if slaves can fill researcher jobs, or at least I hope these optimizations are already implemented! Only the devs know and exploring the source code can reveal that.
Note 2: Colonies with low numbers of pops make this calculation cheaper.
Now there seem to be break points. In my experiments I tried minimizing vacant jobs early but it seems that you must be thorough and only once the number of free jobs is very small the game speeds up. You have to take into account the AI colonies as well here. It may be that:
a. Caching is there and helps up to a specific point then the engine gives up and dies.
b. Data structures that assist in job calculations suffer as they are becoming larger, so if you cross a threshold as free jobs are reduced and they get downsized, things become exponentially faster.
c. other unaccounted engine stuff that I can’t know about.
)
