• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Stellaris Dev Diary #149 - Technical improvements

Hi everyone, this is Moah. I’m the tech lead on Stellaris and today I’m here to talk about the free 2.3 "Wolfe" update that will be arriving together with Ancient Relics, and what it brings to the table in terms of tech.

Stellaris is going 64 bits.
People have been clamoring for this for a while now, and various factors have led us to finally do this for this patch. I should temper your expectations though: while many have claimed that this would be a miracle cure for all their issues with Stellaris, the reality is somewhat more tame.

What does it mean?
The one solid benefit is that Stellaris is no longer limited to 4gb of memory, and won’t crash anymore in situations where it was reaching that limit. For people who play on huge galaxies, with many empires, many mods or well into 3000s, this will be a boon.

In terms of performance, though, it doesn’t change much. Without drowning you in technical details, let’s just say that some things go faster because you handle more data at once, some things go slower because you have more data to handle. In the end, our measurements have shown no perceptible difference.

Finally, the last effect of switching to 64 bits is that the game will no longer playable on 32 bits computers or OSes. We don’t think this will affect many people, but there you have it.


What about Performance?
I know that’s everyone’s favourite question, so let’s do our best to talk about it. First, let me dispel some notions floating around in various forums: Stellaris does use multithreading, and we’re always on the lookout for new things to thread. In fact between 2.2.0 and 2.2.7, a huge effort was made to thread jobs and pops, and it’s one of the main drivers of performance improvement between these version.

Pops and jobs are indeed what’s consuming most of our CPU time nowadays. We’ve improved on that by reducing the amount of jobs each pop evaluate. We’ve also found other areas where we were doing too much work, and cut on:
  • Ships calculating their daily regeneration when they’re at full health
  • Off-screen icons being updated
  • Uninhabitable planets doing the same evaluations as populated planets
Why do these seemingly pointless things happen? Well, we generally focus on getting gameplay up and working quickly so that our content designers can iterate quickly, and sometimes things fall through the cracks. Some of these systems are also quite complex and the scale of the new code is not so easily apparent. Sometimes, not limiting the number of targets is good enough because you’re not doing much but then, months later, someone adds more calculations or the number of objects explodes for unrelated reasons, and suddenly you’ve got a performance issue.

Modifiers
One thing that sets Stellaris apart from other PDS title is how much we use (or abuse) modifiers. Everything is a modifier. Modifiers are modified by other modifiers themselves modified by other modifiers, and sometimes by themselves. It’s quite hard to follow, and leads to every value being able to change at any time without your noticing.

“Why don’t you just compute jobs when a new one appears?” has often been asked around these parts. Well, a short answer to that is it’s really hard to know when a new job appears. You can get jobs from any modifier to: country, planet, pops. Each of these can get modifiers from ethics, traditions, perks, events, buildings, jobs, country, planets, pop, technology, etc.

Until now we were trying to calculate modifiers manually, forced to follow the chain in its entirety: when you recompute a country modifier, you then calculate their planets modifiers, and then each planet would recalculate their pops modifiers. Some of our freezes were just that tangled ball of yarn trying to sort itself out.

NexRiPkna2utTqAzF9H0DEjOCwHVsI4EejYO-vMQMh6QwUB-_uP7dXmpjkwXzOOKoiwDqkSzd9tlLmN3DlFN2R06A62od6XxWm8xh99XRDfRFRP3vVj42GBIaDaXSK7jjyKdS39b

This is our modifier flow charts. It’s not quite up to date, but gives you an idea of the complexity of the system (Unpolished because it’s a dev tool, and not made for the article).

No More!
For 2.3 “Wolfe” we have switched to a system of modifier nodes, where each node register what node they follow, and is recalculated when used, following the chain itself. We have modifiers that are more up to date, and calculated only when needed. This also reduces the number of pointless recalculations.

This system has shown remarkable promise, and cut the number of “big freezes” happening around the game (notably after loading, for example). It has some issues, but as we continue working with it, it’ll get better and help both with performance and our programmers’ sanity.

So, what’s the verdict?
In our tests, 2.3 “Wolfe” is between 10% and 30% faster than 2.2.7 right now. Hopefully it’ll stay that way until release, but the nature of the beast is that some of these optimizations break things and fixing the issues negate them, so we can’t promise anything.

IuIGuQ4cXPvjCEMWG_AowiNIFXhzpsPIcphmCVJD79vQqVMqUeZCqCoVfDlWDNZ3YNkAScYAJh2ebft947YsqoOhG7A_4pNBWxjZ6L9se5lkEEImNYZ4uOpTMWj-amEiwSYdirpd


Measurements provided by @sabrenity , using detailed info from the beta build. It’s worth noting the “SHIPS_SERIAL” purple line has since been eliminated.

AI
Another forum favorite, we have done some improvements to the AI. First, with @Glavius ’s permission, we’ve used his job weights to improve general AI job distribution. We’ve also done the usual pass of polish and improvements, and of course taught the AI how to use all our new features.


What else is new?
We’re also getting a new crash reporter that will send your crash report as soon as they happen rather than next time you start the game. We’ve improved our non-steam network stack for connectivity issues, etc.


All right, enough of my yammering. This has turned into a GRRM length novel, and even though there are many more areas we could cover, we’ll just turn this for your perusal.
 
Last edited:
  • 1
Reactions:
32-bit user here. Earlier in thread someone asked why can't people like me buy a new pc or do something else in order to move to 64-bit. My answer is that i don't have any money due to the fact that i'm a 17 year old student, living in Russia, who's going to have the exams soon, and later (probably) go into an university, which means that i won't have any money to buy a new pc for several years. It'll be a miracle if i'll be able to find those money.:(

If you want to know characteristics of my computer, then there they are: CPU: AMD A8-3870 APU with Radeon(tm) HD Graphics 3.00 GHz; Installed memory: 6,00 Gb (2,74 Gb available); System type: 32-bit operating system.
 
With all things considered, the move to 64-bit and all, will save games from 2.2 still work in 2.3 or not?
 
32-bit user here. Earlier in thread someone asked why can't people like me buy a new pc or do something else in order to move to 64-bit. My answer is that i don't have any money due to the fact that i'm a 17 year old student, living in Russia, who's going to have the exams soon, and later (probably) go into an university, which means that i won't have any money to buy a new pc for several years. It'll be a miracle if i'll be able to find those money.:(

If you want to know characteristics of my computer, then there they are: CPU: AMD A8-3870 APU with Radeon(tm) HD Graphics 3.00 GHz; Installed memory: 6,00 Gb (2,74 Gb available); System type: 32-bit operating system.

That CPU should be capable of running a 64-bit OS. Have you tried?
 
32-bit user here. Earlier in thread someone asked why can't people like me buy a new pc or do something else in order to move to 64-bit. My answer is that i don't have any money due to the fact that i'm a 17 year old student, living in Russia, who's going to have the exams soon, and later (probably) go into an university, which means that i won't have any money to buy a new pc for several years. It'll be a miracle if i'll be able to find those money.:(

If you want to know characteristics of my computer, then there they are: CPU: AMD A8-3870 APU with Radeon(tm) HD Graphics 3.00 GHz; Installed memory: 6,00 Gb (2,74 Gb available); System type: 32-bit operating system.
Pretty sure that CPU is 64bits so if you have Windows you just have to install the 64 bits version since the key works for both and that is easy to do.

Also that is a pretty good computer you have there*, no sense in even considering changing for years unless you have money to waste or it breaks.

*Several times better than mine for instance, the CPU alone is two thirds of the price I paid way back.
 
32-bit user here. Earlier in thread someone asked why can't people like me buy a new pc or do something else in order to move to 64-bit. My answer is that i don't have any money due to the fact that i'm a 17 year old student, living in Russia, who's going to have the exams soon, and later (probably) go into an university, which means that i won't have any money to buy a new pc for several years. It'll be a miracle if i'll be able to find those money.:(

If you want to know characteristics of my computer, then there they are: CPU: AMD A8-3870 APU with Radeon(tm) HD Graphics 3.00 GHz; Installed memory: 6,00 Gb (2,74 Gb available); System type: 32-bit operating system.

CPU is fully capable of 64-bit, and the same install key should work for the 64-bit version of the OS. Granted, you have to reformat/reinstall, but that should be all you have to do if you still have your CD key.
 
The move to 64 bits is the occasion to get rid of many overflow bugs that plague the game. Either by using bigger variables, or by setting caps to prevent the values to overflow and become negative.
Examples:
Weapons range overflow prevents whole fleets from firing.
Research points overflow prevents your empire from building anything.
etc.

32-bit programs aren't limited to 32-bit variables or chunks of memory. 64-bit programs just have an easier time manipulating variables / memory chunks that are greater than 32-bits in size (ie it speeds up performance slightly in those particular cases where 64-bit variables are used)
 
To be honest I do not know why you have that cache and calculate it all at once...
*you have your map, all stars - these are nodes.
*each node is connected to some other nodes and distance is defined for each connection
*for gate to gate distance is 0 or very small amount - they are special nodes
*some nodes are more important than others - ala chokepoints.
*when you build a graph of such important nodes number of them decreases significantly.

(...)

additionally:
*distance is bad - you should calculate by time needed to traverse from system A to B and add time needed for a fleet to go through the system (sublight speed) and number of jumps (cooldown counts) and node distances in a star system from one entry point to the other.
*it doesn't have to be perfect path - it has to be close enough - no one will notice!
*if you are doing cache with all distance for all stars you are calculating it all in one moment why? why do sth 5000x ? why not calculate it on the fly when the distance is needed?

if you use on demand pathfinding calculation new things will be possible for Stellaris - like:
* hyperlanes with different base speeds or even one directional (A>B but not B>A)
*hyperlanes that canot be used till certain tech level (aka distances... or simplified warp...)
*hyperlanes that can be used only by certain empire or group of empires (aka old womholes...)
*unlimited hyperlane creation and destruction that can reshape galaxy map
*some structures could modify time multiplayer for hyperlanes (aka inhibitors - 10x slower travel time... or faster)
all you need is store nodes and have additional parameters for each node connection like distance, speed modifier, list of restrictions/allowed techs/empires).

I am no coder, and I might be wrong but this is what people do in the internet to calculate distances/time.

Regards,
TC

Exactly, most time is taken by sub-light travel via star systems (between hyperlane entrances +cooldown), so number of jumps decides most how long travel will take.
Wormholes/Gateways bypass many jumps at once, so with plenty gateways it may even feel like good ol' warp sometimes ;)

Gateway cap should be stricter than starbase cap (+upkeep), and 'hard' disable option, so you may conquer system with gateway, reinforce your fleet, then disable it, and opponent may retake system, and rebuild it again ;)
 
And here I am still hoping I can get conditions for my habitat mod that check the object the habitat/moon is orbiting, rather than needing to use events to create a work around. :D

All the improvements to the code seem good, though!
 
Will the update enable even larger galaxies? I know that there are mods out there to increase the number of stars but the "size" seems to be static so it looks crowded
 
I've suggested removing gateways/wormholes/l-gates but for some reason nobody likes it when i suggest we remove features. Go figure!
Please don't remove them. Removing a feature is by far the worst thing a developer can do. This was the exact reason I almost stopped supporting Stellaris; FTL features got removed.
There is NEVER a valid reason to remove a feature. You could just as easily make a config or options setting that can be set to enable/disable the feature.
In the case of gateways and such, there is a slider on game start settings screen that can be set to 0, so that no gateways or wormholes appear. Perhaps a similar setting could also be made to disable their relevant techs so that no empire could build their own.
If a given player finds the performance cost too great, they can simply disable the techs and set their spawn to 0. Done.
This is a win-win. They can be disabled by default; providing a performance boost, while players who prefer to keep them can enable them.
A very obvious solution that almost all developers seem to somehow miss, almost always resulting in a significant loss of players (and therefore potential income).
From a business perspective, this is also a win because the feature still exists and can easily be enabled and used, therefore keeping players who prefer the feature from leaving; making them more likely to spend more money on future content (especially given they don't have to fear that content they paid for getting removed).
 
Our issue is that we have a cache that contains the distance from any system to any other system, and when you add/remove bypasses or systems we need to recalculate this cache. This is further compounded by the fact not everyone has the same access to every bypass.

Why you cannot store a distance to closest accessible gateway for each system? It is a lot less data, and can be calculated with one breadth-first search from "good" gateways, which is linear in time and not quadratic. And distance between two arbitrary systems is easy to calculate this way: take minimum of normal hyperlane distance and sum of distances towards nearest gateway.
 
Tnanks for reply and telling me that my computer is capable of using 64-bit programs, except it's not... I don't know the reason, but i can't play games like EU 4 (i had the ability to play it earlier) or Cities Skylines. Maybe it is due to the fact that my pc claims to be from 2009 (despite the fact that we bought it in 2011?)



I'm not really into computers from technician view, but my computer constantly crashes every time after playing a game for around 10 minutes. Is there something that can help me? (when crashing, the monitors creates thin lines.)
 
Tnanks for reply and telling me that my computer is capable of using 64-bit programs, except it's not... I don't know the reason, but i can't play games like EU 4 (i had the ability to play it earlier) or Cities Skylines. Maybe it is due to the fact that my pc claims to be from 2009 (despite the fact that we bought it in 2011?)
As you mentioned already you are running a 32-bit operating system. You need to change that. Then you'll be able to use all your RAM as well instead of only a fraction like now.
 
Please don't remove them. Removing a feature is by far the worst thing a developer can do. This was the exact reason I almost stopped supporting Stellaris; FTL features got removed.
That'll teach me making jokes, I guess.

With all things considered, the move to 64-bit and all, will save games from 2.2 still work in 2.3 or not?
As always, we try to preserve compatibility, but can't guarantee anything.
 
Maybe it is due to the fact that my pc claims to be from 2009 (despite the fact that we bought it in 2011?)
Probably came pre-installed with an older version of Windows then. If you have a legit Windows key you can select between 32 Bit or 64 Bit when installing

The best would be to get a cheap version of Windows 10. And then do a clean install. Not an upgrade. Windows 7 is an option too though
 
Last edited:
@Guraan
Could you get a significant boost by ditching the necessity to calculate optimal routes for long distances? If we're sending fleets half across the galaxy most people won't even notice if the calculated routes was a couple jumps longer then an optimal one. To use a real world map as example, if you wanted to go from Stockholm to Berlin, according to google maps you're using a route through Jönköping. But if we came up with one through Göteborg instead it wouldn't be too bad. Only local routes, for example within a cluster would need to be optimal.