• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
I am yet to be convinced the issue is the volume of processing required, and not unnecessary calls to reoptimise the same jobs and the same pops again and again.

As I understand it .... each pop has a preference for the stratum it wants to work in (rights), and a weight to measure how good it is at certain jobs (traits). All other factors (tech, policy, empire ethics, etc) affect all pops in an empire or on a planet equally and shouldnt influence relative job weights.

The only time jobs should be reoptimised is when a new pop arrives on a planet, or the species design or rights are changed. This doesnt happen multiple times per month.

What am I missing here?

You are not missing anything. We just can't tell from the outside and the devs are not talking.
 
The only time jobs should be reoptimised is when a new pop arrives on a planet, or the species design or rights are changed. This doesnt happen multiple times per month.

What am I missing here?

You're missing the time when a new job is created as it can be better to take an already employed pop for this one rather than an unemployed pop and use the latter for the old job. Which happens approximatively at the same rate. :p
 
You're missing the time when a new job is created as it can be better to take an already employed pop for this one rather than an unemployed pop and use the latter for the old job. Which happens approximatively at the same rate. :p

Yup. Missed construction. Still not convinced it explains the issue.
 
Opening and closing jobs.
Enabling and Disabling buildings (you might think that's the same as above, but it's not).
Prioritizing jobs.
Demolishing buildings or districts.
Buildings becoming ruined by population decrease (from purging, bombardment, etc.)
Modding pops to change their traits and thus job weights.
Terraforming the (inhabited) planet to change habitability and thus weights relative to other pops (I don't actually know if habitability applies weight penalties but ever since 2.3.0 it should).
Events that cause jobs to appear (Odd Factory, Dimensional Portal, etc.).
Clearing blockers causing jobs to appear (the spire on relic worlds).
Adopting traditions causing jobs to appear (Prosperity finisher being the obvious example, but there are more).
Events that cause pop traits to change (Worm-in-Waiting being the big one).
Ascension perks causing traits to change (Mind over Matter, etc.)
Secret projects causing traits to change (Flesh is Weak, etc.)
Distributing luxuries raising amenities enough that it's no longer urgent to have as many amenity-producers as possible.
Another farm on some far-off planet coming online and turning a deficit into a surplus, so it's no longer urgent to have as many farmers as possible.
Reforming government and changing civics or authority in a way that changes what jobs are available.
Ethics shift that changes production of various jobs.
Changing planet designation should also affect job weights (though I'm not sure it does).
Researching a tech that changes production could (should?) shift weights.
Researching a tech that makes some pops now allowed to work jobs they weren't before (Droids, Synths, etc.)
Declaring an edict/campaign/ambition that changes production.
Changing a governor or ruler (either the whole leader or just a trait on them) in a way that affects production or priorities.
Factions changing happiness causing crime to change and thus prioritizing police and/or creating criminal jobs.

...

There are a lot of things that mess with pop jobs, or mess with weights. They can broadly be bucketed in those categories, so you could try to trigger job re-evaluation only then, but it's nonetheless kind of a lot and you'd probably miss some... and it's still going to have a hard time catching conditional changes to job weights.
 
AnxYDoF.png




lir1tWm.png


What am I doing wrong? I only selected one Mod. I fear that this is what is crashing my game.
 
...
Distributing luxuries raising amenities enough that it's no longer urgent to have as many amenity-producers as possible.
Another farm on some far-off planet coming online and turning a deficit into a surplus, so it's no longer urgent to have as many farmers as possible.
....

These are the two (types of) events that bother me the most. I somewhat get that the AI needs to do a lot of job tailoring to prioritise its economy to shifting demands or objectives, but it seems to be out of control and swinging wildly from one extreme to another way more often than it should (Unless its just my poor impression).

There are a lot of things that mess with pop jobs, or mess with weights. They can broadly be bucketed in those categories, so you could try to trigger job re-evaluation only then, but it's nonetheless kind of a lot and you'd probably miss some... and it's still going to have a hard time catching conditional changes to job weights.

As you said, all of the things you mentioned fall into a few broad categories ... things that change the number of jobs, things that change the job weights, and things that change the pop rights. i.e., they wouldn't all directly trigger a job optimisation, but may do so indirectly. (e.g., a species change triggers a change in job weights, which triggers a change in job optimisations on selected planets). On systems I have worked on, we have always sought to design triggers to affect the bare minimum number of records, and only applied a catchall fix on very rare occassions (if at all).

There are a few things I think should not affect job weights. These are hab(?), happiness, faction membership, etc. A pop with mining traits should *always* get the mining job, irrespective of happiness, hab, faction membership, housing, etc. To my mind, these are modifiers that should be applied to the job output, and should not be considered as input factors in job selection. I know this might result in a sub-optimal output, but that is kind of the point, otherwise you wouldnt bother to fix hab, happiness, etc.
 
1. No you don't. Stellaris conditions can be transformed into boolean expressions over boolean-valued conditions. Boolean expressions can be transformed into a normal form. Calculating normal form with variables known can be implemented using cycles only, without if-then-else's .
Could you demonstrate this approach? I am curious. Perhaps PM to avoid side-tracking the thread. Although even if it's possible the number of instructions per cycle tell us that Stellaris is not doing it.

To verify my sanity I took one of the job assignment scripts and coded it in C++. I've simulated 5K pops and run the loop evaluating that script (in compiled form):
Code:
1.5e+10
Pop count = 5000
Number of evaluations = 3000000
Elapsed time = 55.913s


      55914.091550      task-clock (msec)         #    1.000 CPUs utilized         
                16      context-switches          #    0.000 K/sec                 
                 0      cpu-migrations            #    0.000 K/sec                 
               197      page-faults               #    0.004 K/sec                 
   214,086,955,557      cycles                    #    3.829 GHz                      (37.50%)
    78,505,273,517      stalled-cycles-frontend   #   36.67% frontend cycles idle     (50.00%)
   524,448,303,594      instructions              #    2.45  insn per cycle         
                                                  #    0.15  stalled cycles per insn  (62.51%)
   180,050,196,534      branches                  # 3220.122 M/sec                    (62.51%)
         3,534,678      branch-misses             #    0.00% of all branches          (62.51%)
   195,210,335,515      L1-dcache-loads           # 3491.255 M/sec                    (49.97%)
    15,023,616,988      L1-dcache-load-misses     #    7.70% of all L1-dcache hits    (25.00%)
       189,736,943      LLC-loads                 #    3.393 M/sec                    (25.00%)
   <not supported>      LLC-load-misses                                             

      55.914489264 seconds time elapsed
Here we have 2.45 instruction per cycle (more or less in the normal for the compiled code).

For the curious, the code is here:
Code:
#include <iostream>
#include <cstdlib>
#include <vector>
#include <chrono>

enum Traits {
   trait_mechanical                       = (1 << 0),
    trait_robust                           = (1 << 1),
   trait_robot_efficient_processors       = (1 << 2),
};

enum Flags {
    racket_species_flag                    = (1 << 0),
};

enum class Jobs : uint16_t {
    foundry
};

class Pop
{
public:
    uint64_t        traits;
    uint64_t        species_flags;
    bool            has_citizenship_rights = true;
    bool            can_take_servant_job = false;
    bool            has_owner = true;
    bool            is_enslaved = false;
    bool            is_being_purged = false;
    bool            is_being_assimilated = false;
   
    Jobs            current_job;
    char            reserved[40];           // to simulate memory requirement for some other attributes
   
    inline bool hasAnyTrait(uint64_t requiredTraits) const { return (traits & requiredTraits) != 0; }
    inline bool hasSpeciesFlag(uint64_t requiredFlages) const { return (species_flags & requiredFlages) != 0; }
};

bool jobs_work_minerals = false;
int years_passed = 100;

inline float conditional_modifier(float value, bool condition) { return (condition ? value : 1.0); }

float compute_weight_modifier(Pop const& pop)
{
    if (pop.has_owner && !pop.is_enslaved && !pop.is_being_purged && !pop.is_being_assimilated)
    {
        float modifier = 1.0
            * conditional_modifier(0.2,pop.has_citizenship_rights && !pop.hasAnyTrait(Traits::trait_mechanical))
            * conditional_modifier(0.1,pop.can_take_servant_job)
            * conditional_modifier(2,pop.hasSpeciesFlag(Flags::racket_species_flag))
            * conditional_modifier(5,pop.current_job == Jobs::foundry)
            * conditional_modifier(5,pop.current_job == Jobs::foundry && pop.hasAnyTrait(Traits::trait_robust|Traits::trait_robot_efficient_processors))
            * conditional_modifier(0,jobs_work_minerals && years_passed > 15)
            ;
        return modifier;
    }
    return 0;
}

int main(int argc, char* argv[])
{
    std::vector<Pop> pops;
    pops.resize(5000);
    double totallyUselessValue = 0.0;
    size_t K = 100;
    if (argc > 1)
        K = std::atol(argv[1]);
    auto start = std::chrono::high_resolution_clock::now();
    for (size_t k = 0; k < K; ++k)
    {
        for (size_t i = 0; i < pops.size(); ++i)
        {
            float modifier = compute_weight_modifier(pops[i]);
//            std::cout << modifier << std::endl;
            totallyUselessValue += modifier;
        }
    }
    auto end = std::chrono::high_resolution_clock::now();
    std::chrono::duration<double> diff = end-start;
       
    std::cout << totallyUselessValue << std::endl;
    std::cout << "Pop count = " << pops.size() << std::endl;
    std::cout << "Number of evaluations = " << K << std::endl;
    std::cout << "Elapsed time = " << diff.count() << "s" << std::endl;
    return 0;
}
There are some guesses about what certain things in the script mean, but the point of the experiment was to show the difference in CPU utilization between the compiled code and PDX scripts.

2. No, typical interpreter code doesn't look like this. This particular part is usually implemented using 'switch' or equivalent which usually compiles into indirect jump.
That was a pseudo-language example showing the logic (depending on the language there are options to optimize dispatching). The main cost is not the operation dispatching, but the instruction bloat which is unavoidable in the interpreter thus leading to the stalls in the front-end
 
There are a few things I think should not affect job weights. These are hab(?), happiness, faction membership, etc. A pop with mining traits should *always* get the mining job, irrespective of happiness, hab, faction membership, housing, etc. To my mind, these are modifiers that should be applied to the job output, and should not be considered as input factors in job selection. I know this might result in a sub-optimal output, but that is kind of the point, otherwise you wouldnt bother to fix hab, happiness, etc.
As far as I know, individual happiness has no impact on pop output anymore; it only affects crime and stability. Since both of those are planet-wide, it doesn't matter. Similarly, faction membership has no individual impact on output. Housing not only has no per-pop impact on job output, it's not even tracked on a per-pop basis; there's simply a planet-wide scaling of stability (and thus production, indirectly) by the ratio of housing required to provided. So, obviously, none of those should affect job selection. There's an argument that less-happy pops should be confined to lower-stratum jobs whenever possible, to keep their political impact low and thus approval (and thus stability, and thus production) high, but the simulationist in me says unhappy pops should, where possible, fight hard for the chance to be heard.

Habitability is different. First of all, some things care about it (resource production) and others don't (defense armies, naval capacity, amenities produced(?), etc.), so a lower-habitability pop should, all else being equal, take the jobs that don't care, rather than a pop that does care. Second, I definitely disagree that pops should ignore anything that impacts production when considering jobs. If I had my way, pops would be matched with jobs on expected output. I'm not sure if there's a computationally feasible way to do this even for the data set of "pops on this planet" - never mind optimizing across the empire - but I don't see any good reason why some factors should be ignored.

To address the reason you gave, I think it's a bad one for two reasons:
  • You still have incentive to get pops' habitability up. It reduces upkeep (which is sort of like increasing production, especially since it does impact amenities and thus stability), it increases growth rate, and it increases production of at least one resource on most jobs; if my low-habitability pop is choosing between entertainer and metallurgist I'd rather they take the entertainer, but I'll still lose some Unity production from their low habitability.
  • It's too general an argument. Once you have gene- or robo-modding, you can in theory optimize almost any pop for a job in some way. Why should you have an incentive (due to suboptimal job assignments) to improve habitability, but not to ensure your slaves are Strong? You should have the same set of incentives in both cases. If you wouldn't argue "you shouldn't prioritize Thrifty pops for trade jobs; you should have incentive to fix the traits of your Merchants", why would you say it for habitability?
 
@SeekingEtermity thanks for your considered response.

1. I too believe that happiness, crime, stability etc are applied globally, and therefore dont affect individual pops, job assignments, which is imo the right way.

2. Hab is a different issue (which I sdmit am in two minds about). If I have a strong pop with tropical hab on a desert planet, competing for a mining job with a weak desert dweller, which one should get the job? I dont believe the game should protect the player from this scenario. If hab was taken into account, the tendency would be to unemploy all pops with bad habs regardless of other traits. The player in this case should take the appropriate action to match the pop to the right hab. That's just my opinion though.
 
It's not going to unemploy most of the low-habitability pops (or any of them, unless there aren't more pops than jobs), just give them lower priority for jobs. If you're getting +15% production from Industrious but -40% production from 20% habitability, you might still be a better miner than anything else but you're a worse miner than a random pop with only a -10% from 80% habitability, and I think the game should consider that. Again, the bad habitability pops will still have jobs (unless you're running a ton of unemployment for other reasons), they just won't have their pick. Why should a mining boss hire somebody who can't stand the heat and keeps needing to take breaks or passing out from heat exhaustion, no matter how dedicated to the general skill of mining, unless there isn't anybody better available?
 
You are not missing anything. We just can't tell from the outside and the devs are not talking.

That's why megathreads exist: to contain discussion to a single topic for easier ignoring, until everyone involved gives up and they can say "it's clearly not that big an issue since nobody cares any more."

It's like shadowbanning with less negative PR.
 
That's why megathreads exist: to contain discussion to a single topic for easier ignoring, until everyone involved gives up and they can say "it's clearly not that big an issue since nobody cares any more."

It's like shadowbanning with less negative PR.
Well, we can always just stop buying their content, while using other social platforms for open discussions which can't be "contained" that way.
 
It's not going to unemploy most of the low-habitability pops (or any of them, unless there aren't more pops than jobs), just give them lower priority for jobs. If you're getting +15% production from Industrious but -40% production from 20% habitability, you might still be a better miner than anything else but you're a worse miner than a random pop with only a -10% from 80% habitability, and I think the game should consider that. Again, the bad habitability pops will still have jobs (unless you're running a ton of unemployment for other reasons), they just won't have their pick. Why should a mining boss hire somebody who can't stand the heat and keeps needing to take breaks or passing out from heat exhaustion, no matter how dedicated to the general skill of mining, unless there isn't anybody better available?

Exactly why I am in two minds about it.

Argh .. anyway ... i am quite frustrated at the moment, having just given up on a game because the pop management frustrated the hell out of me. I did a random scan of my planets (circa 2350) and not one single planet was growing my carefully crafted and genetically engineered founder species, but was growing all kinds of random pops and hybrids. Was faced with having to genetically engineer dozens of different species, or enable purging and pop controls and suck up the happiness hit.

Next time i swear I am playing a despotic slaver again, so I can purge the crap out of anything that isn't blue and tentacled! So much easier when your founder species are rulers and scientists, droids do the specialist work, and slaves do the menial work.
 
I read somewhere that Stellaris doesn't utilize CPU multi-threading, is this true? If it is that could seriously speed up the game...
No, it's not true. Stellaris does utilize CPU multi-threading.

Using it has not seriously sped up the game.

The game design requires many functions to still operate on single-threads, in a queue, for game-state synchronisation/linearity, due to reliance on one function finishing before another may begin. Otherwise the game's mechanics would break. It runs in real-time and is not turn-based.
 
Last edited:
That's why megathreads exist: to contain discussion to a single topic for easier ignoring, until everyone involved gives up and they can say "it's clearly not that big an issue since nobody cares any more."
Alternative point of view is that megathreads make it easier for developers (and users) to follow the topic...
 
No, it's not true. Stellaris does utilize CPU multi-threading.

Using it has not seriously sped up the game.
Yes, Stellaris is using multi-threading and it clearly shows in the stats.

The game design requires many functions to still operate on single-threads, in a queue, for game-state synchronisation/linearity, due to reliance on one function finishing before another may begin. Otherwise the game's mechanics would break. It runs in real-time and is not turn-based.
When you think about it this doesn't seem to an inherent problem in Stellaris. For example, pop evaluation on each planet could be done in parallel. In most (if not all) scenarios there's no contention for any data at all.