• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Stellaris Dev Diary #149 - Technical improvements

Hi everyone, this is Moah. I’m the tech lead on Stellaris and today I’m here to talk about the free 2.3 "Wolfe" update that will be arriving together with Ancient Relics, and what it brings to the table in terms of tech.

Stellaris is going 64 bits.
People have been clamoring for this for a while now, and various factors have led us to finally do this for this patch. I should temper your expectations though: while many have claimed that this would be a miracle cure for all their issues with Stellaris, the reality is somewhat more tame.

What does it mean?
The one solid benefit is that Stellaris is no longer limited to 4gb of memory, and won’t crash anymore in situations where it was reaching that limit. For people who play on huge galaxies, with many empires, many mods or well into 3000s, this will be a boon.

In terms of performance, though, it doesn’t change much. Without drowning you in technical details, let’s just say that some things go faster because you handle more data at once, some things go slower because you have more data to handle. In the end, our measurements have shown no perceptible difference.

Finally, the last effect of switching to 64 bits is that the game will no longer playable on 32 bits computers or OSes. We don’t think this will affect many people, but there you have it.


What about Performance?
I know that’s everyone’s favourite question, so let’s do our best to talk about it. First, let me dispel some notions floating around in various forums: Stellaris does use multithreading, and we’re always on the lookout for new things to thread. In fact between 2.2.0 and 2.2.7, a huge effort was made to thread jobs and pops, and it’s one of the main drivers of performance improvement between these version.

Pops and jobs are indeed what’s consuming most of our CPU time nowadays. We’ve improved on that by reducing the amount of jobs each pop evaluate. We’ve also found other areas where we were doing too much work, and cut on:
  • Ships calculating their daily regeneration when they’re at full health
  • Off-screen icons being updated
  • Uninhabitable planets doing the same evaluations as populated planets
Why do these seemingly pointless things happen? Well, we generally focus on getting gameplay up and working quickly so that our content designers can iterate quickly, and sometimes things fall through the cracks. Some of these systems are also quite complex and the scale of the new code is not so easily apparent. Sometimes, not limiting the number of targets is good enough because you’re not doing much but then, months later, someone adds more calculations or the number of objects explodes for unrelated reasons, and suddenly you’ve got a performance issue.

Modifiers
One thing that sets Stellaris apart from other PDS title is how much we use (or abuse) modifiers. Everything is a modifier. Modifiers are modified by other modifiers themselves modified by other modifiers, and sometimes by themselves. It’s quite hard to follow, and leads to every value being able to change at any time without your noticing.

“Why don’t you just compute jobs when a new one appears?” has often been asked around these parts. Well, a short answer to that is it’s really hard to know when a new job appears. You can get jobs from any modifier to: country, planet, pops. Each of these can get modifiers from ethics, traditions, perks, events, buildings, jobs, country, planets, pop, technology, etc.

Until now we were trying to calculate modifiers manually, forced to follow the chain in its entirety: when you recompute a country modifier, you then calculate their planets modifiers, and then each planet would recalculate their pops modifiers. Some of our freezes were just that tangled ball of yarn trying to sort itself out.

NexRiPkna2utTqAzF9H0DEjOCwHVsI4EejYO-vMQMh6QwUB-_uP7dXmpjkwXzOOKoiwDqkSzd9tlLmN3DlFN2R06A62od6XxWm8xh99XRDfRFRP3vVj42GBIaDaXSK7jjyKdS39b

This is our modifier flow charts. It’s not quite up to date, but gives you an idea of the complexity of the system (Unpolished because it’s a dev tool, and not made for the article).

No More!
For 2.3 “Wolfe” we have switched to a system of modifier nodes, where each node register what node they follow, and is recalculated when used, following the chain itself. We have modifiers that are more up to date, and calculated only when needed. This also reduces the number of pointless recalculations.

This system has shown remarkable promise, and cut the number of “big freezes” happening around the game (notably after loading, for example). It has some issues, but as we continue working with it, it’ll get better and help both with performance and our programmers’ sanity.

So, what’s the verdict?
In our tests, 2.3 “Wolfe” is between 10% and 30% faster than 2.2.7 right now. Hopefully it’ll stay that way until release, but the nature of the beast is that some of these optimizations break things and fixing the issues negate them, so we can’t promise anything.

IuIGuQ4cXPvjCEMWG_AowiNIFXhzpsPIcphmCVJD79vQqVMqUeZCqCoVfDlWDNZ3YNkAScYAJh2ebft947YsqoOhG7A_4pNBWxjZ6L9se5lkEEImNYZ4uOpTMWj-amEiwSYdirpd


Measurements provided by @sabrenity , using detailed info from the beta build. It’s worth noting the “SHIPS_SERIAL” purple line has since been eliminated.

AI
Another forum favorite, we have done some improvements to the AI. First, with @Glavius ’s permission, we’ve used his job weights to improve general AI job distribution. We’ve also done the usual pass of polish and improvements, and of course taught the AI how to use all our new features.


What else is new?
We’re also getting a new crash reporter that will send your crash report as soon as they happen rather than next time you start the game. We’ve improved our non-steam network stack for connectivity issues, etc.


All right, enough of my yammering. This has turned into a GRRM length novel, and even though there are many more areas we could cover, we’ll just turn this for your perusal.
 
Last edited:
  • 1
Reactions:
We haven't worked on that (yet). Unfortunately this is a problem for which we don't have a good solution yet.
Our issue is that we have a cache that contains the distance from any system to any other system, and when you add/remove bypasses or systems we need to recalculate this cache. This is further compounded by the fact not everyone has the same access to every bypass.
We have the "basic" cache which is only for hyperlan distances, and then we have a cache patch that adds distance through gateways accessible to that country. This country specific cache needs to be emptied whenever a bypass gets added, and towards the end game, every country starts building gateways, leading to mass cache invalidations and reconstructions.
Add to that that, invariable, the pathfinding itself becomes more complicated because you get many more ways to reach the same point.
Until we find a genius idea, i'm not sure we can do much to improve that. I've suggested removing gateways/wormholes/l-gates but for some reason nobody likes it when i suggest we remove features. Go figure!

Have you considered a cap on the number of gates an empire can built, similar in concept to a starbase cap? I notice in past games that when gate tech is unlocked production across the board expands pretty quickly as AI and human players will build lots of them, so a game will go from having the fixed generated amount to a rapidly expanding network mid to late game. So aside from the empire economy, there's not really a limit or speed bump to the technical cost gates impose. This cap does not need to be a hard cap, the cap can be extended via tech and ascension perk, and it can be balanced in such a way that gates will be extremely prized given their overall rarity.
 
Last edited:
It would be great if difficulty settings would affect AI that way the easy settings gives you AI that cant handle economy (like at the moment), medium is just good at it (like i hope it will be on 2.3), and hard od min-maxing everything like in some mods.
AI agressivness slider is ok.
But there should be separated slider "handicap" giving more resources to players->everyone->AI.

But anyway, great job, i cant w8 how Stellaris will change.
 
Have you considered a cap on the number of gates an empire can built, similar in concept to a starbase cap?
Uh, yeah, adding to this!

In my current game, as a vassal-taking Barbaric Despoiler, there is a #%@$ING SCARY Driven Assimilator that controls upwards of 50% of a 1000 star galaxy. Seemingly every month they finish a new gateway.

The end result of this is that I can NEVER EVER EVER build a single gateway, otherwise all the new 'borders' running between our empires will tank our Defensive-pact opinion from +100 to -1000 through border friction alone in an instant. And I really do NOT want to fight, well, that.
 
Why are you throwing away the whole cache? You only need to recalculate the systems with a new shortest lane to an accesible bypass. A list that grows shorter with every added bypass. In fact, why don't you just cache the shortest distance to the Gateways/L-Gates. You need to jump through a little hoop when the shortest distance involves both a Gateway and an L-Gate, but nothing hard computationally.

I think the issue is that, with the ability to build Gateways, the system has to handle arbitrary amounts of recalculations. Adding a Gateway can, theoretically, change the shortest distance calculation (presumably the heart of the pathfinding system) of every system pair.

Now, for Wormholes, since you cannot artificially create them (I think?), something you could do is the following:

1) At game start, cache every graph of every possible combination of active Wormholes, and create a reference to it.
2) Each Player holds a reference to one of these graphs.
3) When pathfinding is called, just send the reference that Player currently has.
4) When a Wormhole is explored, change the reference the Player holds.

This would increase the game creation time, but should shorten in-play delay by greatly reducing the recalculations needed for wormhole exploration. (It also could be applied to precursor home systems, I think, if those are generated at game creation and not dynamically.)

As for Gateways, that's a more daunting issue. The memory footprint necessary to do my above suggestion for arbitrary numbers of Gateways is likely (literally) astronomical. Even searching the cache for the new reference would likely cause a similar delay to normal.

The two main methods for improving performance here is getting a faster algorithm--which may be possible, but presumably something they are already doing--or improving the cacheing. Likely also being done.

I'm willing to offer my help, as I've had to do quite a bit of pathfinding algorithm work on my current simulation modeling projects.
 
Funnily enough, killing 75% of all pops would resolve our performance issues.
What's stopping you? Not counting purge and occasional orbital bombardment (you did reduce % of planetery population dying to that for no reason), pops could start dying to hunger, crime, and who knows what else? Right now overpopulation is number one problem late-game, so why not make it easier for pops to die? I don't see any downsides, provided it will come with a proper game mechanic.
 
Have you considered a cap on the number of gates an empire can built, similar in concept to a starbase cap?
Putting in my support to this. This may also help them become more strategic, and something you have to plan for, rather than something you can spam once you have the resources.

EDIT (Also add upkeep for them)
 
I've just explained why this happens, and we don't have a good solution right now.


There's some improvement, but we weren't able to dedicate as much time to this as we wanted.
Aren't those kind of events negligible, it's the daily tick that grinds after a while. If it took a minute to generate a new system 7 times across an entire playthrough, I'd be content if generally things were smoother
 
Putting in my support to this. This may also help them become more strategic, and something you have to plan for, rather than something you can spam once you have the resources.

EDIT (Also add upkeep for them)
IMO, upkeep is always a reasonable solution, caps - not so much (especially with things that can't be destroyed yet can be captured).
 
As much as I enjoy the dev diaries about new content/features, these technical dev diaries are what I really like. More like this, please.
 
So we have an idea for next Galactic Crisis! A Space Flu!
Kills 75% of empire population, and reshuffles the cards on the table ( 75% totally, but not equally! ) :D
I've been wanting a space flu kind of crisis, forcing nations to cooperate to solve an intergalactic extinction level event without fighting for a while. It's unfortunately not in the plans right now.
 
I've been wanting a space flu kind of crisis, forcing nations to cooperate to solve an intergalactic extinction level event without fighting for a while. It's unfortunately not in the plans right now.
Sounds like it could work with/after future diplomatic improvements!

@HugsAndSnuggles
IMO, upkeep is always a reasonable solution, caps - not so much (especially with things that can't be destroyed yet can be captured).

Fair, but it might be good if you could destroy them. I just suggested an upkeep on top of that so it wouldn't be as easy to go over.
 
Might be a bit offtopic, but would it be possible to change the Launcher to include a "disable all mods"-button (which should also work on mods that aren't downloaded yet).
I recently got a new computer, but it has a bad wifi connection and setting up the Lan connection takes longer then planned.
This means that my stellaris mods aren't downloaded yet, but the Launcher does try to enable them, letting the game crash on start. I cannot disable them befire they are downloaded which means i will have to wait about 3 days of pure download time to play stellaris again.
 
I've been wanting a space flu kind of crisis, forcing nations to cooperate to solve an intergalactic extinction level event without fighting for a while. It's unfortunately not in the plans right now.
Would be interesting if it could be initiated by some genocidal machine intelligence. :)