• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Stellaris Dev Diary #149 - Technical improvements

Hi everyone, this is Moah. I’m the tech lead on Stellaris and today I’m here to talk about the free 2.3 "Wolfe" update that will be arriving together with Ancient Relics, and what it brings to the table in terms of tech.

Stellaris is going 64 bits.
People have been clamoring for this for a while now, and various factors have led us to finally do this for this patch. I should temper your expectations though: while many have claimed that this would be a miracle cure for all their issues with Stellaris, the reality is somewhat more tame.

What does it mean?
The one solid benefit is that Stellaris is no longer limited to 4gb of memory, and won’t crash anymore in situations where it was reaching that limit. For people who play on huge galaxies, with many empires, many mods or well into 3000s, this will be a boon.

In terms of performance, though, it doesn’t change much. Without drowning you in technical details, let’s just say that some things go faster because you handle more data at once, some things go slower because you have more data to handle. In the end, our measurements have shown no perceptible difference.

Finally, the last effect of switching to 64 bits is that the game will no longer playable on 32 bits computers or OSes. We don’t think this will affect many people, but there you have it.


What about Performance?
I know that’s everyone’s favourite question, so let’s do our best to talk about it. First, let me dispel some notions floating around in various forums: Stellaris does use multithreading, and we’re always on the lookout for new things to thread. In fact between 2.2.0 and 2.2.7, a huge effort was made to thread jobs and pops, and it’s one of the main drivers of performance improvement between these version.

Pops and jobs are indeed what’s consuming most of our CPU time nowadays. We’ve improved on that by reducing the amount of jobs each pop evaluate. We’ve also found other areas where we were doing too much work, and cut on:
  • Ships calculating their daily regeneration when they’re at full health
  • Off-screen icons being updated
  • Uninhabitable planets doing the same evaluations as populated planets
Why do these seemingly pointless things happen? Well, we generally focus on getting gameplay up and working quickly so that our content designers can iterate quickly, and sometimes things fall through the cracks. Some of these systems are also quite complex and the scale of the new code is not so easily apparent. Sometimes, not limiting the number of targets is good enough because you’re not doing much but then, months later, someone adds more calculations or the number of objects explodes for unrelated reasons, and suddenly you’ve got a performance issue.

Modifiers
One thing that sets Stellaris apart from other PDS title is how much we use (or abuse) modifiers. Everything is a modifier. Modifiers are modified by other modifiers themselves modified by other modifiers, and sometimes by themselves. It’s quite hard to follow, and leads to every value being able to change at any time without your noticing.

“Why don’t you just compute jobs when a new one appears?” has often been asked around these parts. Well, a short answer to that is it’s really hard to know when a new job appears. You can get jobs from any modifier to: country, planet, pops. Each of these can get modifiers from ethics, traditions, perks, events, buildings, jobs, country, planets, pop, technology, etc.

Until now we were trying to calculate modifiers manually, forced to follow the chain in its entirety: when you recompute a country modifier, you then calculate their planets modifiers, and then each planet would recalculate their pops modifiers. Some of our freezes were just that tangled ball of yarn trying to sort itself out.

NexRiPkna2utTqAzF9H0DEjOCwHVsI4EejYO-vMQMh6QwUB-_uP7dXmpjkwXzOOKoiwDqkSzd9tlLmN3DlFN2R06A62od6XxWm8xh99XRDfRFRP3vVj42GBIaDaXSK7jjyKdS39b

This is our modifier flow charts. It’s not quite up to date, but gives you an idea of the complexity of the system (Unpolished because it’s a dev tool, and not made for the article).

No More!
For 2.3 “Wolfe” we have switched to a system of modifier nodes, where each node register what node they follow, and is recalculated when used, following the chain itself. We have modifiers that are more up to date, and calculated only when needed. This also reduces the number of pointless recalculations.

This system has shown remarkable promise, and cut the number of “big freezes” happening around the game (notably after loading, for example). It has some issues, but as we continue working with it, it’ll get better and help both with performance and our programmers’ sanity.

So, what’s the verdict?
In our tests, 2.3 “Wolfe” is between 10% and 30% faster than 2.2.7 right now. Hopefully it’ll stay that way until release, but the nature of the beast is that some of these optimizations break things and fixing the issues negate them, so we can’t promise anything.

IuIGuQ4cXPvjCEMWG_AowiNIFXhzpsPIcphmCVJD79vQqVMqUeZCqCoVfDlWDNZ3YNkAScYAJh2ebft947YsqoOhG7A_4pNBWxjZ6L9se5lkEEImNYZ4uOpTMWj-amEiwSYdirpd


Measurements provided by @sabrenity , using detailed info from the beta build. It’s worth noting the “SHIPS_SERIAL” purple line has since been eliminated.

AI
Another forum favorite, we have done some improvements to the AI. First, with @Glavius ’s permission, we’ve used his job weights to improve general AI job distribution. We’ve also done the usual pass of polish and improvements, and of course taught the AI how to use all our new features.


What else is new?
We’re also getting a new crash reporter that will send your crash report as soon as they happen rather than next time you start the game. We’ve improved our non-steam network stack for connectivity issues, etc.


All right, enough of my yammering. This has turned into a GRRM length novel, and even though there are many more areas we could cover, we’ll just turn this for your perusal.
 
Last edited:
  • 1
Reactions:
There is already a trade-off. Less pops.
No, there isn't, because at the point where you use Pop Controls you don't WANT more pops. More pops would just sit around being unemployed and overcrowded and miserable.
And if all that planets are, trouble, then the game should do sth to fix that.
So what you're saying is the game should just let you grow up to a billion pops per planet without any downsides, and the player shouldn't have to worry about it at all. Whether you MEAN to say that or not, that's what you are saying.
Because the game should be helping the player help themselves.
Yeah, and it does, by giving you Pop Controls.
Its use is sth that should be encouraged to help decrease micro.
Yeah, and it encourages its use, by making you not get Sad Overcrowded Unemployed pops.
With disincentives it does the opposite.
Do you also look at the influence cost of edicts and go 'Disincentive to use edicts'? Do you look at the alloy cost of a battleship and go 'this disincentives me to build a fleet'?

Because I get the feeling you don't really know the difference between 'trade off' and 'incentive/disincentive'.
Disincentivizing that is bad game design.
I'm sorry, but after what you've said, I don't trust your word on what is or isn't bad game design.
 
No, there isn't, because at the point where you use Pop Controls you don't WANT more pops. More pops would just sit around being unemployed and overcrowded and miserable.
Or they could be used to quickly develop new colonies. Resettlement is still a thing.

So what you're saying is the game should just let you grow up to a billion pops per planet without any downsides, and the player shouldn't have to worry about it at all. Whether you MEAN to say that or not, that's what you are saying.
No, it is not.

Yeah, and it does, by giving you Pop Controls.
And disincentivizes you from using them.

Yeah, and it encourages its use, by making you not get Sad Overcrowded Unemployed pops.
Sad overcrowded pops that are a very useful resource for expansion.

Do you also look at the influence cost of edicts and go 'Disincentive to use edicts'? Do you look at the alloy cost of a battleship and go 'this disincentives me to build a fleet'?
No. At least not when I can afford them.

Because I get the feeling you don't really know the difference between 'trade off' and 'incentive/disincentive'.
You are trading a stab hit to avoid microhell. Yep, sounds like you are given some great options there.

I'm sorry, but after what you've said, I don't trust your word on what is or isn't bad game design.
Oh noes. What a tragedy. How will I sleep tonight?
 
Resettlement is still a thing.
Yes, and if you don't want to bother, that's what the controls are for. To not make yourself miserable with micro to squeeze out the last 0.5% efficiency.
No, it is not.
Of course you'd say that.
Sad overcrowded pops that are a very useful resource for expansion.
Again, if you want to make yourself miserable with micro for a minuscule boost in efficiency.
You are trading a stab hit to avoid microhell. Yep, sounds like you are given some great options there.
You are VASTLY overestimating the penalty of -5 stability to call it a 'stab'.
Oh noes. What a tragedy. How will I sleep tonight?
I mean, I imagine you'll sleep poorly, since it obviously bothers you enough to keep responding and keep trying to convince me you're right.
 
Yes, and if you don't want to bother, that's what the controls are for. To not make yourself miserable with micro to squeeze out the last 0.5% efficiency.
It's a lot more than 0.5%. A LOT more. Pop growth is basically the number 1 factor in developing colonies and it takes years. But being able to utilize the growth provided by several other colonies increases that growth multi fold.

Again, if you want to make yourself miserable with micro for a minuscule boost in efficiency.
Wouldn't it be nice if the game gave me incentives to nudge me the other way huh? Instead of disincentives... just saying...

You are VASTLY overestimating the penalty of -5 stability to call it a 'stab'.
Well stab is short for stability really. But its importance I would say varies. In my playstyle with all my nobles it doesn't matter much. But a disincentive is still a disincentive.

I mean, I imagine you'll sleep poorly, since it obviously bothers you enough to keep responding and keep trying to convince me you're right.
If that's what you think I'm doing you are quite wrong. When someone really believes sth you can almost never convince them otherwise, particularly on the internet. I do care about the topic though and I will present my opinion, perhaps someone less opinionated on this subject will find it interesting.
 
Last edited:
We have thought about doing "clusters" pathfinding, but haven't had time to look into them.

What if you ditched calculating "distances" (ie travel time) and only calculated the number of jumps? I could imagine this would be a much easier calculation. This wouldn't be accurate, but it would be "close enough" that most won't even notice. Alternatively, you could change the game rules such that all jumps take the same amount of time (regardless of distance) which actually makes sense from a physics sense as the galaxy is already not to scale; and you could also allow all jumps to take place anywhere outside a galaxy well (rather than specific spots) and increase the jump recharge to compensate.

On top of that, you could also use existing choke points to simplify things further, in that you only need the optimal path between all choke points (which is at least a 80% lower number than the total number of systems), and then the optimal path within each cluster. If a wormhole or gate is present, just add those systems to the list of chokes.

Could you also make it more expensive to build gateways? For example have them cost a large upkeep, or alternatively require them to operate in pairs (but for simplicity, allow multiple gates to pair with one single hub gate, if desired).

Also, how much would eliminating the need to "explore" a wormhole help?
 
Nah that is the hyperlane cache, that one is only recalculated if you add or remove a hyperlane... Rest is the "patch" cache and that one is not a 1:1 mapping but on demand.


Yes that is the big question isnt it O;)

yes it is. To be honest still don't know why you need all distances for jour pathfinging algorithm.

I understands that distances are needed for AI to strategize: what systems are of interest, how far is enemy fleet etc.
but for pathfinding all you need is a data structure with all connections and distances. It could be different data structure (simplified), that is updated less comon than "distance cache" which could be then updated with less priority.
It would be interesting to know what benefits the game have for Distance cache.
If pathfinding is fast and there still is a problem I would look for optimizations elsewhere and not is code but in game rules.
For example : why AI calls pathfinding so often? does it really need to do it that often? maybe not.
Why AI builds so many gates? maybe it is a design flaw - maybe gates should be limited and not "spam" all galaxy.
Why AI calls distance cache for strategizing, does it really need distances? maybe number of jumps would suffice. if something is 5 umps away it is less important than 1 jump away from border.
 
Our stats show less than 1% of users are still on 32 bit systems. The intent is never to abandon anyone, but ultimately tech needs to move on to allow further development, and our other option was to abandon all Mac users, period. That is not going to be done.

Also as there has been quite good support for older versions of the game as well (in that they have been kept available to the players) perhaps you can keep one 32 bit version available for people with ancient machines? Now most likely that version then won`t get anymore new updates, but at least then there would be something to use for that 1 % of players as well before they update their systems to this decade. :)
 
Last edited:
@Alastor You should try the Automatic Pop Migration mod. It is amazing how much better the late game is when you have the proper tools to direct the flow of unemployed pops automatically. No change in balance either. You still pay the resettlement cost for each pop moved.

https://steamcommunity.com/sharedfiles/filedetails/?id=1617534169&searchtext=Automatic+Pop+Migration

Yes, and if you don't want to bother, that's what the controls are for. To not make yourself miserable with micro to squeeze out the last 0.5% efficiency.

Would it kill you if there was a vanilla way to automate resettlement for those of us who want to use it without adding 10,000 extra clicks to each game? You don't seem to mind about controls existing to help you NOT micro your empire, so don't you think it is a touch hypocritical not want controls in place for people on the other end of the spectrum?
 
@Alastor You should try the Automatic Pop Migration mod. It is amazing how much better the late game is when you have the proper tools to direct the flow of unemployed pops automatically.

https://steamcommunity.com/sharedfiles/filedetails/?id=1617534169&searchtext=Automatic+Pop+Migration



Would it kill you if there was a vanilla way to automate resettlement for those of us who want to use it without adding 10,000 extra clicks to each game?
I know you think you're being reasonable, but you're really not. The follow-up to this argument is "We should have a vanilla feature for EVERYTHING any minority of players could EVER want. After all, it won't kill those who don't use it!" but this ignores the developer time needed to do it.

An auto-resettler on its own would be simple. But then you need to make it check for energy costs, else people will complain "Stupid autoresettler put me in the red!" and you need to make it check for habitability else "Stupid autoresettler is putting my people on a tomb world when there's already uplifted cockroaches breeding there!" and then you need to check for job-suitability else "Stupid autoresettler is putting my excess Thinker Pops onto my mining world!" etc etc
 
I know you think you're being reasonable, but you're really not. The follow-up to this argument is "We should have a vanilla feature for EVERYTHING any minority of players could EVER want. After all, it won't kill those who don't use it!" but this ignores the developer time needed to do it.

An auto-resettler on its own would be simple. But then you need to make it check for energy costs, else people will complain "Stupid autoresettler put me in the red!" and you need to make it check for habitability else "Stupid autoresettler is putting my people on a tomb world when there's already uplifted cockroaches breeding there!" and then you need to check for job-suitability else "Stupid autoresettler is putting my excess Thinker Pops onto my mining world!" etc etc

And yet, someone already did it with a mod months ago.

https://steamcommunity.com/sharedfiles/filedetails/?id=1617534169&searchtext=Automatic+Pop+Migration

Imagine for a moment that there was no vanilla way to stop pop growth on your planets. You could still play ping-pong with force growth and effectively stop pop growth all together on all planets, but it would be a lot of extra clicks. In this situation, I believe you would think that adding a way to stop pop growth on a planet with a decision would be a novel, even necessary, idea. Forced stopping of pop growth and automated resettlement are two sides of the same coin. To say that only your preferred side has any merit is selfish and short-sighted, but you're right, I'm the one being "unreasonable" here.

After it's all said and done, the fact remains that Resettlement is a feature of the game. No one is forced to use it, but if it is going to exist, it might as well exist in such a way that it isn't a nightmare to use.

"Stupid autoresettler is putting my excess Thinker Pops onto my mining world!" etc etc

The game already can't figure out how to properly distribute jobs based off pop traits, an auto-resettler or lack thereof won't change that.
 
And yet, someone already did it with a mod months ago.
And how much time did they need to do it? How many 'cheaty' methods did they use in the code that Paradox themselves cannot? (Glavius AI at one point had AI satisfy their 'urge to build buildings' with a cheap instant effectless building that instantly destroyed itself, for instance).
Imagine for a moment that there was no vanilla way to stop pop growth on your planets. Y
It's pointless to imagine because that is not the reality we live in.
The game already can't figure out how to properly distribute jobs based off pop traits, an auto-resettler or lack thereof won't change that.
Have to disagree there. I see a proper pop-trait distribution all the time. Resilient Pops, for instance, always seem to end up as Soldiers and Enforcers.
 
And how much time did they need to do it? How many 'cheaty' methods did they use in the code that Paradox themselves cannot? (Glavius AI at one point had AI satisfy their 'urge to build buildings' with a cheap instant effectless building that instantly destroyed itself, for instance).

Of all the mods that could be used as a strawman for why mods are "cheaty" in their methods, Glavius AI is the last one I ever expected to see be used. The mod is such a success that the devs are actually using some of its code in the next update. Also, if its stupid, and it works, then its not stupid. With that said, one of the merits of bringing a feature to vanilla is that, in theory, no one understands the code better than the devs, and therefor, no one is more able to smoothly integrate a feature without the use of "cheaty" methods into the game.

It's pointless to imagine because that is not the reality we live in.

This is a non-argument. Every feature of this game at one point was not a part of reality and existed only in the imagination of one of the devs. Progress begins when someone looks past the way something is and imagines the way that something should be.

Have to disagree there. I see a proper pop-trait distribution all the time. Resilient Pops, for instance, always seem to end up as Soldiers and Enforcers.

My Intelligent Miners, Industrious Technicians, and Very Strong Entertainers solute you, for you have done what I could not.

If this is indeed the norm, and my experience with the game has thus far been a fluke, wouldn't that make your previous argument that an auto-resettler would mess with coherent job-distribution a moot point, if indeed the game is apt enough to sort them out?
 
Of all the mods that could be used as a strawman for why mods are "cheaty" in their methods, Glavius AI is the last one I ever expected to see be used.
And yet there it is. That's the method it used, whether you want to deny it or not.
e mod is such a success that the devs are actually using some of its code in the next update.
And I'll bet you anything the 'instantly vanishing building' isn't part of it.
Progress begins when someone looks past the way something is and imagines the way that something should be.
Except that's not what you did. You didn't look at the way something should be, you looked into a world that lacks things we have, and imagined a solution that world needs, and then tried to say "Therefore, that is what OUR world needs."
If this is indeed the norm, and my experience with the game has thus far been a fluke, wouldn't that make your previous argument that an auto-resettler would mess with coherent job-distribution a moot point, if indeed the game is apt enough to sort them out?
Perhaps it would be a moot point. Or perhaps the auto-resettler would, for some reason, but unable to use the job-assigning code. After all, if it picks unemployed pops to resettle, you get one issue. If it picks up employed pops, it'll have to, well, compare every single pop across every single one of your planets to every single open job slot across every single planet and compare THAT to the potentially-opened job slots you get post resettle.

A very different beast than just swapping out a Soldier and a Technician on a single planet.

But I want to back up to this:
To say that only your preferred side has any merit is selfish and short-sighted, but you're right, I'm the one being "unreasonable" here.
Please get off your high-horse; you're no different than me in this regards. If either of us believed the 'other side' had merit, then we would not be defending our current positions, but rather the other.

We'll just have to agree to disagree.
 
Also as there has been quite good support for older versions of the game as well (in that they have been kept available to the players) perhaps you can keep one 32 bit version available for people with ancient machines? Now most likely that version then won`t get anymore new updates, but at least then there would be something to use for that 1 % of players as well before they update their systems to this decade. :)
All versions before first 64-digit will remain 32-digit. Those will be available via beta tab in Steam, or you could just keep a copy of the version you need separately from Steam/GOG/PDX Launcher auto-update folder.
 
And yet there it is. That's the method it used, whether you want to deny it or not.

And I'll bet you anything the 'instantly vanishing building' isn't part of it.

When did I deny Glavius used such a method? I also remember that that method got the job done, as unorthodox as it was. You seem to be more concerned that an unorthodox method could be used to automate resettlement. Would a method that is smoothly integrated without the use of "cheaty" methods then be ok with you?

Except that's not what you did. You didn't look at the way something should be, you looked into a world that lacks things we have, and imagined a solution that world needs, and then tried to say "Therefore, that is what OUR world needs."

If you lack the ability to see the correlation between the scenario I presented and the one we are debating now, I cannot help you.

Resettlement already exists. I'm talking about improving an existing feature. Remember when the Fleet Manager didn't exist? No sane person would argue that the old way of fleet creation was better than it is today. The devs saw that an existing feature was inconvenient to use, so they improved it by giving it a better interface and reducing the number of required clicks. 2.3 will be adding a way to automate planets via pre-made templates, and the devs have stated they are exploring the possibility of players being able to make their own templates. You know what that is? Automation, and you aren't even required to use it. Conceptually, there is no difference between automated planet construction and automated pop resettlement.

Perhaps it would be a moot point. Or perhaps the auto-resettler would, for some reason, but unable to use the job-assigning code. After all, if it picks unemployed pops to resettle, you get one issue. If it picks up employed pops, it'll have to, well, compare every single pop across every single one of your planets to every single open job slot across every single planet and compare THAT to the potentially-opened job slots you get post resettle.

A very different beast than just swapping out a Soldier and a Technician on a single planet.

And then the planet would sort the jobs for pops based off their traits? Either the game can intelligently assign jobs to pops or it cannot, which is it?

Please get off your high-horse; you're no different than me in this regards. If either of us believed the 'other side' had merit, then we would not be defending our current positions, but rather the other.

If the devs announced that they were thinking about removing the ability to stop pop-growth on individual planets, I would be one of the first ones to call them out on it. I don't use it because I prefer to maximize pop growth at all times, but I see the merit in the ability to do so. I like to min-max my efficiency, you do not, and both methods are equally valid ways to enjoy the game. The difference between us is that I am ok with everyone having their own convenient tools and you are not.

We'll just have to disagree.

Indeed.
 
Last edited:
I know you think you're being reasonable, but you're really not. The follow-up to this argument is "We should have a vanilla feature for EVERYTHING any minority of players could EVER want. After all, it won't kill those who don't use it!" but this ignores the developer time needed to do it.

An auto-resettler on its own would be simple. But then you need to make it check for energy costs, else people will complain "Stupid autoresettler put me in the red!" and you need to make it check for habitability else "Stupid autoresettler is putting my people on a tomb world when there's already uplifted cockroaches breeding there!" and then you need to check for job-suitability else "Stupid autoresettler is putting my excess Thinker Pops onto my mining world!" etc etc

I'm already annoyed enough about clearing blockers eating energy out of the central pool rather than "resources" from the poois I've given to the sectors.
Especially when I'm trying to get energy credits together for a decision (like the galactic market) or to buy alloys for quick reinforcement. Or even just to avoid a temporary dip in energy income sending me red.

And yeah... I can see "stupid resettlement bot moved my Thinker pops to a mining world where they became enslaved, and promoted miners to science jobs and they now won't demote, and all whilst I was building science buildings for those thinker pops!" becoming a complaint.
 
I have a question for Devs, but the passage from 32bit to 64bit how does it will end up about Modding? And another, does it will mean we'll have to do all the settings of the game again about Graphics and what not?
 
Our stats show less than 1% of users are still on 32 bit systems. The intent is never to abandon anyone, but ultimately tech needs to move on to allow further development, and our other option was to abandon all Mac users, period. That is not going to be done.

Thanks for not abandoning Mac users! We are few but we are loyal!
 
I dont necessarily micro my pops to the last detail, but I do use the resettle interface to fill up new colonies especially ringworlds or city worlds, and to balance slaves and servants across suitable planets.

For those purposes I do find the current resettle feature painful. I do believe the interface can be improved to show more pertinent information or to allow for movement of multiple pops at a time or to more easily identify planets with excess pops or a preponderance of the desired species.

E.g.planet list to show total jobs and total pops, total servants, total slaves, current crime and stability.
Instead if job lists, show species list grouoed by category (citizens, servants, slaves, robots) with hab and traits, and allow for moving of multiple pops per click.

Maybe just me, but I find it tricky to identify planets by which species are prevalent on them. The only reasonable place is the sector screen, but you have to expand each sector to see which planets have large numbers of which species.

I do use auto pop migration to sort out the micro that I dont otherwise bother with, but it's the macro issues I am concerned about (slaves, servants, etc.)
 
Hello
I know it may not have a lot of common with this Dev Diary, but I had an idea to add into Stellaris.
I have been playing Stellaris for a year and a half now, and I think that megastructures, apart from the enormous bonuses they provide, do not offer a lot of gameplay. So I have thought about adding a colony like view to megastructures. When I thought about them, the first thing that came to my mind was how many people can fit in there, and in the example of a the scientific megastructure, where do all the scientists that work in there come from ? So,(without including ringworlds and habitats) I have thought about transforming all megastructures into special planet types, retaking the example of a scientific megastructure, with a specific job type( in this case scientists ). I think it will add more realism to megastructures and a new range of possibilities with them.
Thanks for considering my idea.