Has the attempt to improve computing power failed?

[Arkarnia]

I think that the game still has the same number of calculations, only now they are grouped together instead of individually. Instead of individual residents, we now have the same number of population groups on the planet, and these groups also have different sizes. You see, the jobs still have a number that is used by the groups and the individual sizes of these groups. Obviously, individual calculations are still used for this.

I would change the districts so that the population is simply a counter that shows the number of inhabitants on the planet, and the efficiency of all the districts on this depends. If the number is 100%, then it also performs 100%. So if the planet has a capacity of 4,000 population and there are only 2,000 population, then the planet with all the districts is only working at 50%. This means that the planet takes a pro rata of the number of population compared to the capacity and then calculates the percentage as the efficiency of the districts.

 

[HFY]

Some of the groups I see are small, fewer than 100 pops total.

I'm honestly curious if the maximum number of groups per colony is smaller than the 3.x number of pops per colony.

 

[grommile]

I'm honestly curious if the maximum number of groups per colony is smaller than the 3.x number of pops per colony.

The interesting question for me is not the maximum (because it's trivial to create a degenerate scenario), but the median and the upper quartile.

In this context, lowering the automigration hab threshold to 20% was probably a mistake.

 

[Falrinn]

The new pop group system tends to work best with large planets where the pop groups average more than 100 apiece. I'm not sure where the break even point is, but the greatest benefit over the old system will be in the late game when performance matters a lot.

Being somewhat less efficient in the early game isn't as big of a deal.

 

[Abdulijubjub]

The new pop group system tends to work best with large planets where the pop groups average more than 100 apiece. I'm not sure where the break even point is, but the greatest benefit over the old system will be in the late game when performance matters a lot.

Being somewhat less efficient in the early game isn't as big of a deal.

This is my thought as well. The fact that there are as many pop groups in the early/mid game as there were pops in older versions doesn't matter compared to there being much fewer pop groups in the late game (when performance is precious) compared to pops before.

Late game planets had 100-200 pops and 100-200 jobs, with the total computations scaling with the product of the two. Now, they have 50-60 pop groups (possibly 100-200, with a ton of distinct species), and maybe a dozen jobs. That's obviously a major savings. That drops by at least 1 order of magnitude, possibly up to 3 orders of magnitude for extremely homogenous societies (synths, purifiers, gestalts).

 

[passeris]

I am gonna laugh so hard if we find out it's pop GROUP calculations causing the lag, and that's why it isn't triggered by 6 million of the same pop. The shitstorm would be cataclysmic.

Well, I didn't think it would happen so soon...

I don't play 4.0 so the following is just speculation on my part. Please correct me where I'm wrong:

Some of the groups I see are small, fewer than 100 pops total.

I'm honestly curious if the maximum number of groups per colony is smaller than the 3.x number of pops per colony.

The maximum number of pop groups in the galaxy should be Species × Strata × Faction × Ethics × Planets, right? Faction is near enough a constant 3. Strata has a value of 3 or 4 (do civilians get counted?). Number of planets has not changed since 3.14. BioGenesis may contribute to an increase in species number, but I don't know how substantially. Ethic is maximum 8, increasing with time from two/three. So the max number of pop groups should be 72 (if not civilians) or 96 (if civilians), multiplied by species, multiplied by planets.

This sounds higher than the max number of pops in 3.14, which was in the region of 100 (regular planet capacity for simplicity's sake) × planets. The difference is that pop groups increase with number of species in the galaxy, and scale with migration, meaning they have the potential to cause a growing burden as the game progresses.

 

[Gilbert95]

To directly answer OP, the pop group change achieved one of the core objectives: improved performance for pops. In late game, where it matters most, there are orders of magnitude fewer pop groups than there were pops in 3.x. There is good evidence that this had the desired effect, including the amusing example of a bug earlier in 4.0.x accidentally creating more than 1,000,000 pops and this had no noticeable impact on performance.

Unfortunately performance has not improved overall because there are other areas creating lag. The primary suspected culprits are ships, fleets and combat.

Performance is one of the key areas devs are working on this summer. However we won't see improvements in the main branch until late August. If you opt into the beta, you'll see some of what they are doing there now.

 

[Drachasor]

Let's remember that they were forced to put this out in a alpha/beta state. It was filled with bugs and clearly a lot of things hadn't been tested. There still are a lot of bugs. This means they haven't tuned the performance at all either.

 

[balmung60]

To directly answer OP, the pop group change achieved one of the core objectives: improved performance for pops. In late game, where it matters most, there are orders of magnitude fewer pop groups than there were pops in 3.x. There is good evidence that this had the desired effect, including the amusing example of a bug earlier in 4.0.x accidentally creating more than 1,000,000 pops and this had no noticeable impact on performance.

Unfortunately performance has not improved overall because there are other areas creating lag. The primary suspected culprits are ships, fleets and combat.

Performance is one of the key areas devs are working on this summer. However we won't see improvements in the main branch until late August. If you opt into the beta, you'll see some of what they are doing there now.

I think I have *slightly* different suspect: pathfinding, particularly with gateways and hypergates, which I'm pretty sure do a lot of weird and unpleasant things to the pathfinding calculations.

Lots of ships is definitely a problem too (and one of the reasons I really don't like corvettes ever being meta - I would rather not have the game's calculations clogged up by thousands upon thousands of tiny little ships), but I think it's the hypergates and gateways that are doing some of the nastiest stuff to performance.

I suspect this because pathfinding is generally pretty computationally heavy and anything that makes it more complicated/dynamic is likely to significantly increase that weight over a simpler and more static map.

 

[Gilbert95]

I think I have *slightly* different suspect: pathfinding, particularly with gateways and hypergates, which I'm pretty sure do a lot of weird and unpleasant things to the pathfinding calculations.

Lots of ships is definitely a problem too (and one of the reasons I really don't like corvettes ever being meta - I would rather not have the game's calculations clogged up by thousands upon thousands of tiny little ships), but I think it's the hypergates and gateways that are doing some of the nastiest stuff to performance.

I suspect this because pathfinding is generally pretty computationally heavy and anything that makes it more complicated/dynamic is likely to significantly increase that weight over a simpler and more static map.

Underlying ships and fleets is pathfinding. In short more ships/fleets = more pathfinding. And as you say bypasses make it worse. But performance is complex and tricky and often there are seemingly innocuous changes that can have unintended consequences.

 

[hart30]

I never really understood why everone was so focused on performance ftom pops. Even inmy 3.14 games lategame performance massively tanked. And the end game crisis murdering over half the galaxy with all its pops made it worse instead of better.
It could only have been the number ships/fleets or the crisis fighter AI. Everything else was either destroyed or greatly mimimized.

 

[ArpoMo6]

To directly answer OP, the pop group change achieved one of the core objectives: improved performance for pops. In late game, where it matters most, there are orders of magnitude fewer pop groups than there were pops in 3.x. There is good evidence that this had the desired effect, including the amusing example of a bug earlier in 4.0.x accidentally creating more than 1,000,000 pops and this had no noticeable impact on performance.

Unfortunately performance has not improved overall because there are other areas creating lag. The primary suspected culprits are ships, fleets and combat.

Performance is one of the key areas devs are working on this summer. However we won't see improvements in the main branch until late August. If you opt into the beta, you'll see some of what they are doing there now.

But we still haven't seen a single example of performance impact reduction from pops. The developers haven't provided us with any performance profiling results that would show a reduction in the impact of pops on performance (and have they done any at all?). Just words.

I wouldn't be surprised if the current average number of population groups in 4.0.x is comparable to the average nubmer of pops in 3.14 and we ended up with no positive impact.

That example with several million population during the uprising of the machines is not a good example, because it generated only one population group with one ethics (i.e., in translation to 3.14, this is exactly one pop).

Until clear "before" and "after" pops performance impact graphs are provided, all claims that the developers have fixed the population issues are nothing more than speculation.

 

[StragaSevera]

Unfortunately performance has not improved overall because there are other areas creating lag. The primary suspected culprits are ships, fleets and combat.

So, you mean, they wasted a lot of time and killed the last shreds of balance of the game, because they didn't manage to profile the game? Do I understand you correctly?

 

[Drachasor]

So, you mean, they wasted a lot of time and killed the last shreds of balance of the game, because they didn't manage to profile the game? Do I understand you correctly?

No, they got rid of the biggest bottleneck, and have run into another. But also there is little doubt that there has probably been little or no performance tuning given the state of the game on 4.0's release. It's quite possible things will dramatically improve with a little more effort. Also, sometimes one bottleneck in performance can cover up another. We'll just have to wait and see.

 

[AnCi420]

The current beta is already way smoother on my machine in early late game (but you have to live with the broken intel system and occasional CTDs), looking forward to this weeks changes that will probably bring another boost with the aggregated fleets (and hopefully better late game AI economy due to late game economic plans).

 

[StragaSevera]

> they got rid of the biggest bottleneck, and have run into another
If it was the biggest bottleneck, then fixing it would improve the performance dramatically. If it did not improve performance, then it was not the biggest bottleneck - which they could quickly profile on internal alpha branch to confirm that lowering the amount of pops does not mean the game runs faster.

 

[Abdulijubjub]

So, you mean, they wasted a lot of time and killed the last shreds of balance of the game, because they didn't manage to profile the game? Do I understand you correctly?

> they got rid of the biggest bottleneck, and have run into another
If it was the biggest bottleneck, then fixing it would improve the performance dramatically. If it did not improve performance, then it was not the biggest bottleneck - which they could quickly profile on internal alpha branch to confirm that lowering the amount of pops does not mean the game runs faster.

Suppose 25% of the frame time is spent calculating pops/jobs, 25% is spent calculating fleet pathfinding, and 25% is spent calculating trade route pathfinding (and the remaining 25% is on miscellaneous that's hard to quantify).

You release an update which reduces pop/jobs down to 4/10 of its previous value, and removes trade route pathfinding entirely.

But you (accidentally) inflate the economy by 2x and make the AI more willing to move ships around. And, for multiplayer, you accidentally bring back the missile corvettes meta, which quadruples the number of ships flying around (compared to missile cruisers) and makes engagements longer and more resource intensive. So there's 3x as much ship pathfinding as before: its performance hasn't changed, there's just 3x as much of it.

0.4*0.25+3*0.25+0*0.25+0.25=1.1.

You would have removed or mitigated two of the major bottle necks (previously taking up a full half of the computation time, now only 10%), but a side effect, which can be fixed through game balance in future updates, made the remaining major bottle neck skyrocket. And now performance is 10% worse. But it's now much easier to make better.

At no point in this did you fail to profile, or fail to address issues found through profiling, or do something that would indicate that you missed the original cause of the lag.

The game is still slower, but it's both more easily fixable, and not counterevidence to the claim that the lag was previously (partially) caused by pops and trade routes.

 

[StragaSevera]

You release an update which reduces pop/jobs down to 4/10 of its previous value, remove trade route pathfinding entirely.

But you (accidentally) inflate the economy by 2x and make the AI more willing to move ships around.

Great. You made an internal alpha branch with the changes, and you see that they did not achieve the intended purpose. Then you spend time debugging the new bottleneck that you detected by profiling, instead of sunk-cost-fallacying yourself into the original idea.

At no point in this did you fail to profile, or fail to address issues found through profiling, or do something that would indicate that you missed the original cause of the lag.

You failed to address issues found through profiling when you release game in worse state than before. Because, by definition, if you did address them, then your game wouldn't get this economy inflation.

 

[Apollo1784]

The game's performance will NEVER be in a good state. It has been YEARS on end of lies and changes that do nothing or as in the case of 4.0 literally worsen performance. I'm sure this post will be removed anyways.

 

[Drachasor]

Latest beta patch notes indicate how many fleets (not their size) is a huge performance hit and the main source of current lag. They unfortunately didn't go into details as to why that is, because I'd love to know. But they've increased the fleet command cap a lot to address this.

I'd like to see them address issues with fleets of mixed ships with different AI modules. My unlikely dream is that we even get the ability to simulate fleet combat with different ships. That would be really nice for testing designs and compositions.