• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Stellaris Dev Diary #149 - Technical improvements

Hi everyone, this is Moah. I’m the tech lead on Stellaris and today I’m here to talk about the free 2.3 "Wolfe" update that will be arriving together with Ancient Relics, and what it brings to the table in terms of tech.

Stellaris is going 64 bits.
People have been clamoring for this for a while now, and various factors have led us to finally do this for this patch. I should temper your expectations though: while many have claimed that this would be a miracle cure for all their issues with Stellaris, the reality is somewhat more tame.

What does it mean?
The one solid benefit is that Stellaris is no longer limited to 4gb of memory, and won’t crash anymore in situations where it was reaching that limit. For people who play on huge galaxies, with many empires, many mods or well into 3000s, this will be a boon.

In terms of performance, though, it doesn’t change much. Without drowning you in technical details, let’s just say that some things go faster because you handle more data at once, some things go slower because you have more data to handle. In the end, our measurements have shown no perceptible difference.

Finally, the last effect of switching to 64 bits is that the game will no longer playable on 32 bits computers or OSes. We don’t think this will affect many people, but there you have it.


What about Performance?
I know that’s everyone’s favourite question, so let’s do our best to talk about it. First, let me dispel some notions floating around in various forums: Stellaris does use multithreading, and we’re always on the lookout for new things to thread. In fact between 2.2.0 and 2.2.7, a huge effort was made to thread jobs and pops, and it’s one of the main drivers of performance improvement between these version.

Pops and jobs are indeed what’s consuming most of our CPU time nowadays. We’ve improved on that by reducing the amount of jobs each pop evaluate. We’ve also found other areas where we were doing too much work, and cut on:
  • Ships calculating their daily regeneration when they’re at full health
  • Off-screen icons being updated
  • Uninhabitable planets doing the same evaluations as populated planets
Why do these seemingly pointless things happen? Well, we generally focus on getting gameplay up and working quickly so that our content designers can iterate quickly, and sometimes things fall through the cracks. Some of these systems are also quite complex and the scale of the new code is not so easily apparent. Sometimes, not limiting the number of targets is good enough because you’re not doing much but then, months later, someone adds more calculations or the number of objects explodes for unrelated reasons, and suddenly you’ve got a performance issue.

Modifiers
One thing that sets Stellaris apart from other PDS title is how much we use (or abuse) modifiers. Everything is a modifier. Modifiers are modified by other modifiers themselves modified by other modifiers, and sometimes by themselves. It’s quite hard to follow, and leads to every value being able to change at any time without your noticing.

“Why don’t you just compute jobs when a new one appears?” has often been asked around these parts. Well, a short answer to that is it’s really hard to know when a new job appears. You can get jobs from any modifier to: country, planet, pops. Each of these can get modifiers from ethics, traditions, perks, events, buildings, jobs, country, planets, pop, technology, etc.

Until now we were trying to calculate modifiers manually, forced to follow the chain in its entirety: when you recompute a country modifier, you then calculate their planets modifiers, and then each planet would recalculate their pops modifiers. Some of our freezes were just that tangled ball of yarn trying to sort itself out.

NexRiPkna2utTqAzF9H0DEjOCwHVsI4EejYO-vMQMh6QwUB-_uP7dXmpjkwXzOOKoiwDqkSzd9tlLmN3DlFN2R06A62od6XxWm8xh99XRDfRFRP3vVj42GBIaDaXSK7jjyKdS39b

This is our modifier flow charts. It’s not quite up to date, but gives you an idea of the complexity of the system (Unpolished because it’s a dev tool, and not made for the article).

No More!
For 2.3 “Wolfe” we have switched to a system of modifier nodes, where each node register what node they follow, and is recalculated when used, following the chain itself. We have modifiers that are more up to date, and calculated only when needed. This also reduces the number of pointless recalculations.

This system has shown remarkable promise, and cut the number of “big freezes” happening around the game (notably after loading, for example). It has some issues, but as we continue working with it, it’ll get better and help both with performance and our programmers’ sanity.

So, what’s the verdict?
In our tests, 2.3 “Wolfe” is between 10% and 30% faster than 2.2.7 right now. Hopefully it’ll stay that way until release, but the nature of the beast is that some of these optimizations break things and fixing the issues negate them, so we can’t promise anything.

IuIGuQ4cXPvjCEMWG_AowiNIFXhzpsPIcphmCVJD79vQqVMqUeZCqCoVfDlWDNZ3YNkAScYAJh2ebft947YsqoOhG7A_4pNBWxjZ6L9se5lkEEImNYZ4uOpTMWj-amEiwSYdirpd


Measurements provided by @sabrenity , using detailed info from the beta build. It’s worth noting the “SHIPS_SERIAL” purple line has since been eliminated.

AI
Another forum favorite, we have done some improvements to the AI. First, with @Glavius ’s permission, we’ve used his job weights to improve general AI job distribution. We’ve also done the usual pass of polish and improvements, and of course taught the AI how to use all our new features.


What else is new?
We’re also getting a new crash reporter that will send your crash report as soon as they happen rather than next time you start the game. We’ve improved our non-steam network stack for connectivity issues, etc.


All right, enough of my yammering. This has turned into a GRRM length novel, and even though there are many more areas we could cover, we’ll just turn this for your perusal.
 
Last edited:
  • 1
Reactions:
If you just needed a single path for a single fleet you would indeed be right that calculating on the fly would be fastest. However, my best guess is that we need dozens, by the late game hundreds of paths to be calculated, possibly every tick. On for every seperate fleet or civilian ship. So an algorithm that takes several milliseconds is out of the question, we'Re aiming for microseconds. Hence lookup-tables.
You might be on something their, AI re-evaluating their strategy, planets that want to know how close they are to the capital etc O;)
The pathfinder itself is not slow, it just gets hammered with calls lol
Problem is, that the tables need to be up to date, so you've got an overhead for keeping the tables fresh.
Yep and it also needs to be synchronized between the players in mp as well O;)
Much cred and 2x golden stars @Spartakus
 
Last edited:
Additionally if the cache consist of only distances from any star to any star how this data is useful??
it is not a path through selected stars (nodes) only distance.
An easy solution (not necessarily an efficient one, but I assume that Stellaris uses a variant of it) is to store both the shortest distance to a system b and the outgoing hyperlane used by the shortest path (you will get this info while calculating shortest distance anyway so you might as wlel store it) To then get the actual shortest path you then follow these breadcrums.
 
Ok, but ...
this cache (all distances) is recalculated each time new node is added (so for each distance pathfinding algorithms multiplied by huge number), and in later game it is frequent, so some time gained at first is lost later (and you can feel the slow down in late game)
Nah that is the hyperlane cache, that one is only recalculated if you add or remove a hyperlane... Rest is the "patch" cache and that one is not a 1:1 mapping but on demand.

Additionally if the cache consist of only distances from any star to any star how this data is useful??
it is not a path through selected stars (nodes) only distance.
It is not every path through every node bcs it is "bad idea" ? or not.
and even if it is a path through some nodes, even if there are multiple paths from A>B, some of nodes on those paths could be restricted for this fleet
so no viable paths
then you calculate from scratch.
or maybe it is resolved differently
Yes that is the big question isnt it O;)
 
No, thank you. They're fine, and can be improved.

I think this would also raise a level of outrage, maybe not as high as 2.0, but still a noticable amount. Especially since L-Gates were one of the selling points for Distant stars, and Wormholes/Gateways were a sort of mid/lategame compromise for getting rid of Warp/Wormhole.
I think it might be lost cause to remove Wormholes/Gateways, they were the main reason caused frame drop before 2.0 and it’s why dev made that decision.
We can still keep the L-gate, because their number is small compare to others
 
- Each fleet has different stances that need to be considered if a system is a valid point or not.
- Each system may or may not give you access to pass by ftl inhibitors and your diplo standing with that empire.
- Each wormhole has an requirement that they have to been explored.
Plus
- Each system may or may not be individually marked as restricted by a player

Is this implying that wormholes are being treated as hyperlanes?
Imagine it as a lane with forced zero travel time (well, maybe 1 simulation tick) and w/o graphical line on the map.
As well as gateways are all connected to each other, with permission/restriction flags on those.

Also, isn't 'lane explored' in the case of wormholes just another case of 'can I use this hyperlane'?
Science ships going on auto-exploration could pass through wormholes without exploring of those since technology of wormhole travel is discovered, if systems from both sides of a wormhole are at least explored. Something isn't clear there.

Ok, but ...
this cache (all distances) is recalculated each time new node is added (so for each distance pathfinding algorithms multiplied by huge number), and in later game it is frequent, so some time gained at first is lost later (and you can feel the slow down in late game)
That's (but not only) why that "lookup tables" are separated:
We have the "basic" cache which is only for hyperlan distances, and then we have a cache patch that adds distance through gateways accessible to that country.
 
Much cred and 2x golden stars @Spartakus

Thanks. I wrote the paper for my bachelors degree in graph theory a couple years ago and had little use for it recently, this rant is lot's of fun to me. I wish I could look into this in depth. Just for the fun and educational value, I doubt that I could produce anything that's actually helpful. ;)
 
Thanks. I wrote the paper for my bachelors degree in graph theory a couple years ago and had little use for it recently, this rant is lot's of fun to me. I wish I could look into this in depth. Just for the fun and educational value, I doubt that I could produce anything that's actually helpful. ;)
Well we are not really finished tbh O;P

That's (but not only) why that "lookup tables" are separated:
Its the minimum way to travel, and it is heavily used from scripts both in generation and later on O;)
 
First of all it sounds like this patch will bring some awesome changes. Secondly I've always enjoyed using graphic mods and would like to know if it's possible to make them less likely to break achievements in the future ?

Currently it seems like mods that only affect the UI (Dark UI and the likes) will not break achievements while mods that change species portraits (Human Variety and the likes) will disable them. I get that it's somewhat of a money issue since cosmetic DLC's that adds species portraits can bring in revenue but I still feel these mods really should be achievement compatible since they don´t make them easier to get.
 
Go back to what hapends with a pathfinder like the a* without any optimal heuristics? and since we are lock-step we need 1 and the same solution
 
The solution to the pops and jobs breaking the game is to get rid of both. Simply move resource production to the planets themselves, and have a population/expansion bar that fills up over time, to increase the planet's overall output.
 
The solution to the pops and jobs breaking the game is to get rid of both. Simply move resource production to the planets themselves, and have a population/expansion bar that fills up over time, to increase the planet's overall output.
Then Stellaris really would be a spreadsheet simulator
 
Its the minimum way to travel, and it is heavily used from scripts both in generation and later on O;)
Plus L-gates, gateways and wormholes may not be used at all (IIRC, gateways and wormholes could be switched off via sliders on galaxy generation options screen, Ls wouldn't appear w/o Distant Stars and mods could add their own set of junctions that would require own patch table), so patch tables are addition to the core table.

Go back to what hapends with a pathfinder like the a* without any optimal heuristics? and since we are lock-step we need 1 and the same solution
It would find shortest routes from the start system to all others, as it expands through shortest of available (but not yet used) lanes. Is that how you calculating basic table?
 
What I'd love to see is Pops producing Pops instead planets.
Right now: more planets = king.

It'd also make demographically more sense.

Performance would grind to a halt around the mid-game. If each POP creates its own child POPs then you're looking at an exponential growth factor that would just murder your average CPU after 10 generations.
 
Performance would grind to a halt around the mid-game. If each POP creates its own child POPs then you're looking at an exponential growth factor that would just murder your average CPU after 10 generations.
Oh no sorry. I meant that the more Pops you have, the higher Pop production. Nothing fancy. This would make both Wide and Tall viable and give an advantage over Machine Empires, which would still rely on static manufacturing and its bonuses (BTW I prefer playing Machine Empires).
So in general making Pop production more reliant on Pops instead on the number of planets.
 
Plus L-gates, gateways and wormholes may not be used at all (IIRC, gateways and wormholes could be switched off via sliders on galaxy generation options screen, Ls wouldn't appear w/o Distant Stars and mods could add their own set of junctions that would require own patch table), so patch tables are addition to the core table.
And there we have 2 more gold stars O;)

It would find shortest routes from the start system to all others, as it expands through shortest of available (but not yet used) lanes. Is that how you calculating basic table?
Yes but would it be an a*? or would we just call it a BFS?

Edit: And no thats not how we calc the basic table, would take too long time
 
Last edited:
Most event chains that create a system create it close to the players who finished the chain, though.

If you wish to cut down on hyperlane distance re-caching, I would argue that these event chains that spawn a star system should be re-written to be populated during galaxy creation. This also adds an interesting strategic element to the game.

Let's say you have two empires expanding into an unclaimed section of the galaxy. One empire's science ship stumbles across a Cybrex anomaly (or archaeological dig site in upcoming DLC). An alert can be sent out to other empires stating a Cybrex event has been found in System Y.

If Precursor event chains are pre-established in a region during galaxy creation, we now know there is a potential Ring World in that region of space. No one knows exactly where yet until the event chain is completed, but we know something of extremely high value is there. This would increase the need for local empires to take that region of space faster than other areas they are expanding into. Securing that sector of space becomes a race to also increase a player's chance the Ring World unhidden at the end of the event chain will be off a star system they control.