Performance Megathread

[Riince2]

this is purely anecdotal but while waiting for the poe league to start i had stellaris open and, for fun, i decided to try dragging the login window to my secondary monitor (stellaris on my main monitor) knowing my secondary monitor is powered by the igpu on my cpu. Days slowed down on Stellaris upon doing that, since the igpu shares memory bandwidth with the cpu cores.

I might be hallucinating but this seems to suggest a ram bottleneck. Rip my ddr3 pc...

 

[alexti]

this is purely anecdotal but while waiting for the poe league to start i had stellaris open and, for fun, i decided to try dragging the login window to my secondary monitor (stellaris on my main monitor) knowing my secondary monitor is powered by the igpu on my cpu. Days slowed down on Stellaris upon doing that, since the igpu shares memory bandwidth with the cpu cores.

I might be hallucinating but this seems to suggest a ram bottleneck. Rip my ddr3 pc...

You don't need to guess. Run perf and see if it's a RAM or not. Stellaris is rather heavy on graphics, so it might be due to the performance of GPU or due to performance of main RAM, which is slower than memory on the dedicated video card

 

[UltimateTobi]

You don't need to guess. Run perf and see if it's a RAM or not. Stellaris is rather heavy on graphics, so it might be due to the performance of GPU or due to performance of main RAM, which is slower than memory on the dedicated video card

Stellaris is not heavy on GPU at all. It's a very CPU-heavy game.

 

[alexti]

Stellaris is not heavy on GPU at all. It's a very CPU-heavy game.

Anyone who is interested in knowing the answer can get objective stats observed on their own machines instead of trusting some internet strangers

 

[The Founder]

Most things only get updated once a month anyway - at the start of the next month when you get the combined production of all your planets (energy, minerals, unity science, pop creation, ... ).

Wich would be a good argument. If it was not established that the Job Calculations are producing most of the Background Load.
Moving it into the End of Month tick, would only expand that tick even further.
The game already spreads out the monthly processing:
On the 1st, it is most Economics
On the 2nd, it is Science - including all those new rolls and AI descisions that have to take place, Updating of Designs, etc.
They might have a similar day for Diplomacy. But that might simply be based on Trust ticking up.

Most other Paradox games will not even try to give you current figures. They will only show "last month" measurements. Stellaris tries to break that.

You need to offer an argument rather than simply state that multithreading doesn't work.

Yes, that is true. Wich is why I asked you to provide a Argument for "it is not a problem".

It's completely trivial to invalidate old calculations and run new ones if the situation changes.

Wich of the 60~ ish Ticks you jumped ahead to wll you roll back too? Or will you just store 60 sets of planet data, effectively multiplying the memory demand of planets, pops and everything related by x50?

But let us put your theory to the test. I am going to ask on StackOverflow if rolling back a Alternate thread to a previous state is a remotely good idea.
And if you do already know of the code that works, please share it.

 

[The Founder]

Stellaris is not heavy on GPU at all. It's a very CPU-heavy game.

The graphics run fine on my Integrated Graphics card. That is how little it is stressing the GPU.

All games are limited - inherently - by the performance of the single, main thread.

 

[Less2]

Wich of the 60~ ish Ticks you jumped ahead to wll you roll back too? Or will you just store 60 sets of planet data, effectively multiplying the memory demand of planets, pops and everything related by x50?

Do you think planets need to be evaluated every frame of gameplay? That would be mindbogglingly stupid. Planets and pops on them only affect the global state of the game on the month end tick, that's the whole point.

But let us put your theory to the test. I am going to ask on StackOverflow if rolling back a Alternate thread to a previous state is a remotely good idea.

Here's an idea, maybe you should get education or at least read a book rather than lurk internet forums for people who have no idea what they are talking about. You give the impression of someone stringing technical words together in an attempt to be grammatically correct with no attempt at being relevant. You are completely obsessed with asking trivial, meaningless questions about trivial, meaningless things. You are essentially asking whether allowing a thread to read its own memory is a good idea. Like what? Then you go running off to another site to find someone as ill-informed as you to agree with you and somehow give your argument legitimacy.

Glad to see @Less2 and @BlackUmbrellas dealing with multi-threading again. I really don't want to spend time disagreeing with The Founder, again, or pointing out the game is multi-threaded, again, or that yes parallelisation could be improved, again.

I can see why. It's like trying to talk with a flat-earther who is so sure that they are the intelligent one asking the zinger questions.

 

[sortulv]

The graphics run fine on my Integrated Graphics card. That is how little it is stressing the GPU.

All games are limited - inherently - by the performance of the single, main thread.

Does it run as smoothly when you view large space combats zoomed in? I have noticed that my game tends not to like me running those at high speeds, and as such will always set the speed to slow when viewing combat.

 

[The Founder]

Do you think planets need to be evaluated every frame of gameplay?

I already explained t hat they need to and why they need to.

Here's an idea, maybe you should get education or at least read a book rather than lurk internet forums for people who have no idea what they are talking about.

Here is an idea:
Since you obviously so know much more about this, you show us your Pseudo code?
You know, rather then talking out of your behind saying that everyone that disagrees is "bad wrong fu"?

Let us start with something simple:
How does a game loop - the most fundamental structure of every game on this planet that was not bodged together in a GUI like Windows Forms - look like?

Does it run as smoothly when you view large space combats zoomed in?

Yes if I intentionally increase the load of the Draw code (wich is the little part of drawing that has to run on the CPU right next to Update/Game Ticks), I do slow down overall game performance.
Still a case where the CPU and not the GPU is the bottleneck, however.

 

[AlphaAsh]

Function (run regularly or called, result cached)
Game loop - includes scheduled function calls (regular run).
State check - may have invalidated cached result. Run function again now.

Desynch of game-state becomes more likely the less often function is scheduled, but reduces CPU load, when handed off and relies on state check.

Functions and state checks can be parallelized (run on own threads).

You couldn't get any simpler pseudo-code wise for what @Less2 is trying to tell you.

Replace layman terms with technical terms as wanted.

And whether rendering is not getting dedicated processing on GPU, and is clogging up threads on CPU, is just confusing your arguments.

ClauseWitz's scopes and objects is synonymous with any OO data-implementation. To suggest that the game-loop is currently in its best state for multi-threading, that it is as linear as it has to be, is I feel, nonsense. I believe it can be improved. Your misunderstanding of multi-threading, not withstanding.

 

[The Founder]

You couldn't get any simpler pseudo-code wise for what @Less2 is trying to tell you.

Acutally it is trivial to get simple example/Pseudo code for this:

while( user doesn't exit )
  check for user input
  run AI
  move enemies
  resolve collisions
  draw graphics
  play sounds
end while

Literally from Wikipedia.

This is the core of any serious game devlopment. It runs in single thread. It is the absolute bottleneck of Game Performance.

In practice however you have it cut down to a Update function doing al this:

  check for user input
  run AI
  move enemies
  resolve collisions

And a Draw function:

  draw graphics

Playing sounds? As you mostly hand this off to someone elses code (you just tell them what to play when), you can do those kinds of things in either Update or Draw.

Replace layman terms with technical terms as wanted.

As I am not you, I do not feel any need for doing that.

 

[AlphaAsh]

Irrelevant to what is being discussed, and a level of understanding far lower than what experienced game engineers need to have.

I use layman terms, not technical terms, because I'm not an academic. I'm a professional with enough experience (20+ years) to know it doesn't f*****g matter and that other engineers won't care.

check for user input
  run AI
  move enemies
  resolve collisions

If you think every element of those must be single-threaded, then you need to try making games, rather than just reading about making games. You're wrong. (And hell, that example is for a basic shooter, not a complex 4X. FFS)

 

[AlphaAsh]

Since you love reading about the subject, try reading the right stuff. Read this instead:
https://www.bogotobogo.com/python/M...tithreading_Event_Objects_between_Threads.php
Or post 2 of this:
https://www.gamedev.net/forums/topic/643631-game-thread-synchronization-display-38-game-logic/
Or this:
https://arghya.xyz/articles/thread-synchronization-part-three/
Or:
https://docs.microsoft.com/en-gb/dotnet/standard/threading/overview-of-synchronization-primitives

(at Moderators - sorry if the links are a problem.)

Other software engineers - share some links too. Not bloody wikipedia.

 

[Riftwalker]

And a Draw function:

  draw graphics

.

Okay I just want to make sure everyone knows drawing graphics is almost always a separate thread nowadays. Shaders will keep pumping out new frames even when they aren't given new info unless they're told to wait.

This is why sometimes particle effects will keep playing like laser blasts even if the shops aren't moving or updating position for to a long tick.

At best you need to update the shaders with new pos data and effect calls.

 

[Ashantai]

Settle down, folks. Keep posts on topic and productive please.

 

[Industrious81]

Seemed like reasonable and measured responses to me.

Not sure why the mods are all riled up. Check out some reddit or sports forums if you want to see some real "settle down" type situations.

I have been enjoying the learning process of the last couple pages, as I'm sure others have as well.

 

[Starless2001]

I'm playing a heavily modded version of Shelley (2.5.1) and I took an action that most likely involves going through every pop in the entire galaxy at year 2481: an intense task. So Stellaris stopped responding according to the OS (I hope there's no infinite loop involved and the task will finish soon, it's running while I write this). One thing I noticed, and this is why I'm here, is that only core 6 seems to be overloaded with work, at 100% capacity. The other 11 cores are running normally (at 0 % capacity with the occasional spikes, you know how it is), as if Stellaris wasn't even running at all. It's not a bad design choice to have all those every_* effects implemented via simple iterators with no parallelism or custom allocators if you know Stellaris' code doesn't need it but since there is client code to Stellaris (the mods), maybe a mod could be able to require alternative dlls to be loaded with alternative algorithms, and maybe they should be able to annotate the event that should require the alternative algorithm in some way... It's just a vague idea but maybe the team could have an idea of their own while reading this.

 

[StormyStrife]

"Pops create the most lag. You can't really optimize it much past what is done now"

IMO, partly correct, just let me suggest a debatably simple/complex alternative.

Nothing changes, except all Pops are given an ID number between 1 and 1,000,000 specially for Pops. When a pop is assigned to a job, it never changes jobs on it's own, even if new jobs on a higher stratum are created. Those automatically fill when new pops are created or transferred into your empire, and still abide by limitations (say slaves in a slaver empire can't work certain jobs).
In cases where Slave jobs are already filled on a planet, but there are jobs in a higher stratum available, it'll just display the unemployment icon in the Outliner, players who are actually playing the game will handle it manually, and if using the colony management AI, they will do it instead.

The pops do not randomly shift to other jobs and prioritizing a job only applies to newly created pops in where they're placed first.

What would be nice, is a main menu-type menu where you can order jobs and prioritize ALL jobs in a specific order based on ethics and empire type (mainly normal, Corporate, Machine Empire, and Hive Mind.) with jobs being able to occupy the same slot in a "Equally distribute among these jobs" manner. The weight system could be done for this without any randomness added in like what the weight system usually has to begin with.

I have noticed that pops are randomly shifting at all times, which causes tons of problems, especially when you prioritize low-stratum jobs, and it's probably messing with performance if the AI even prioritizes jobs at all.

I've also noticed that 1 Million is a good number for late and even end-game because pop amounts rarely ever get into the hundred thousands, in fact that's so far past the endgame that you probly don't even have anything to do at that point.

Another idea, is to just ripoff Master of Orion 2, or even 4, and make it so Pops are displayed physically, stacked in the literal, exact same way and you can just pick them up and then "drop" them down into jobs that are available. The max jobs value could be easily made to give hard limits to this, and you'd not change the collapsable menus and keep those.
Jobs with only 2 Jobs could just be made to take up way less space than they do now visually, and only increasing in size as they get larger, with it taking up it's own row after hitting a large number (say... 20)
You'd allow the player to pick up a specific number of pops by clicking at say... the 5th pop from the right of this row would have you pickup 5 pops in that job.
While the pop is picked up, you can freely scroll up and down since you wouldn't need to click and drag, it'd be pretty simple and pop management would be a ton easier for new players this way.

You don't even need to give pops unique portraits, just a very generalized one atleast at first, modders will naturally create better ones anyways.
The problem of "Well what if you put pops down on anywhere that isn't a job" wouldn't exist as you wouldn't be able to, and when you close the planet screen, since you'd already have the Pop ID value, and the assigned job data, the pops held in the cursor would just go back to the jobs they were already assigned.

Assigned job data wouldn't change until they were placed in a new job.
You could simply assign each job type an ID (If they already don't have one) and not do it per job, but in general, with each planet managing how many of each job they have in total, and reporting whatever is actually filled), IE: Planet Omicron has 40 of Job 003 (Mining in this example). Planet Omicron doesn't give each available job an ID, it just knows how many of a job it has, and allows pops to be placed into it up to that max amount of 40 in total.
The AI would be able to simply pull jobs from a less prioritized one into a more needed one instantly, and I could see bugs related to this cropping up, but I still think if you put economy values into the AI scripting (Aka, AI puts jobs from minerals into energy because they have a low energy value, or they have a deficit in energy, and the job used to pull from would be based on whatever they have the most of into whatever they need the most, with Alloys and Research jobs being the highest on the "Keep if absolutely possible" list).

I have no actual idea how it's actually done, but the TL;DR is this.

Pops don't auto filter into jobs on their own, once they get into a Job, they stay there forever unless the AI moves them, or you do, and the AI doesn't move them often like an average or even good Player won't in this system. New pops automatically go into whatever job is most needed. (If you have a deficit, the pops go into jobs to remove the deficit, if not, it works as normal, prioritizing high stratum jobs instead).

Pops could be "picked up" and placed into Jobs and are displayed visually by more than just a number.

Planets wouldn't manage each job individually, just how many they have in total, and how many they have currently.

IMO, if properly implemented, this would probably fix the performance issues related to pops without the need to change much for mechanics.

Honestly, this is probly how it should always be done in a game, I'm sure pops are already given an ID number, if so, it makes literally no sense that they can even move into other jobs automatically.

The problem with stratum demotion in this system could simply be fixed by having a Job yield less while that pop is demoting, and them working in a lower stratum would enable them to demote faster anyways, since that'd make sense, and stratum demotion is really more just a nerf on normal empires anyways.

Stratum demotion wouldn't be so much a restriction as it'd just be a penalty for poor empire management at that point.

i really would love this to be implemented, but being a jaded, cycnical dude, I can't help but feel it'll never happen, even if modded into the game, it probly messes with too many hard-coded things to be workshop-able.

 

[lovenotwar]

I still think the best/easiest answer is to remove individual pops to represent large populations, not only is it not even close to realistic, it creates some terrible mechanics, such as only being able to grow one type of species at a time.
If they moved to using raw numbers we could have realistic population sizes along with multiple species able to grow at the same time, while not removing anything from the planet building aspect.
So pretty much everything would look exactly the same, but instead of the game worrying about multiple pops across multiple species, it would just have to look at singular species. It can even still do a check every day, as it currently does, but it would only be doing it across a few species, over multiple planets, instead of every single pop, on every single planet.

 

[StormyStrife]

Another problem with pop performance might actually be related to job yield calculations. Again, I don't know how it's really done, but it feels like it'd be simpler, and maybe better, to do it like this.

At the start of the game, every owned planet of a Player and AI has job yield formulae assigned to each.

It'd be something like this: Base Yield x Combined total modifiers from planet, racial and tech effects = Per Job Yield. Then it'd just multiply the number of jobs on that planet with that Yield, to get what you're earning.
The game'd never calculate the max number until it's reached, and so you could also make it so the tooltip calls upon that formula's result per Job, to then tell you the "Per Job" yield instead of the total yield, which, given how modifiers work already, is immediately wrong the moment you get a single increase in anything anyways.

Anytime you'd get a tech, or colonize a new planet, or a pre-existing one gets a new modifier, you'd reroll the formula creation for the job that is affected, instead of rerolling all of them. It'd result in empire-wide increases of it and mgiht not lag as hard as empires get bigger.

I could see a possible lag problem arising when you increase all yields by 5% like with Efficient Processors, but at the same time, it'd only happen once.

At present, it feels like the game takes the Base Yield and then multiplies it one by one by every multiplier you have. Like for example you have 5 multipliers for Food on a planet.
The way the game seems to do it (or at least the least efficient way to do it imo) is that it takes the Base Yield, multiplied it by 1.2 for the first 20% increase, takes that result, and repeats everytime using modifiers as a 1.x where x is the percentage converted into a number 2 places down.

I dunno though, take it for all the salt you need. I think simplicity is your best option for resolving the pop issues no matter what they may be.