CK3 Dev Diary #36 - Gotta Go Fast

[Meneth]

All very cool stuff! Any idea how fast this compares to CK2? How many days do you think you would get through in a minute in CK2 on your computer Meneth?

Also any information about how performance progresses throughout the game? ATM CK2 runs really smoothly for the first hundred years for me, but after around 200 it really starts to slow down. I know you've tried to tackle this in the past for CK2, but have the changes in CK3 helped in this regard?

On my specific machine (before the upgrade), CK2 is about as fast as CK3, but with worse framerate and more stuttering.
Haven't tried it after upgrading, but CK3 probably does better thanks to more extensive threading.

Our late game slowdown however I'm relatively sure is less than in CK2.

The part about threading and idk what framrate made me feel incredibly dumb, but if in summary it means a more fluid game, I'm definitely in.

Pretty much!

You should've tried to test the game on machines that below even i5. Not everyone has money to get such PC as you have, even the outdated one.

We have tested the game on i3s. An i3 is part of our current minimum specs on the Steam page.

What do you mean by threading on/off ? It's the hyperthreading activated in the bios ?

Forcing the game's task system to only use a single thread.
This effectively reduces the game to just two threads; the gamestate thread and the render thread.

The traits AI have in CK2 gave them certain hidden values (rationality, zeal, greed, honor, and ambition), so is it basically the same where traits give values like this? If so, what are the hidden values that the AI has now? If it's different can you go into more detail about how?

It's pretty similar. Doesn't quite map 1:1 to CK2, but the idea's the same. The AI personality being displayed in the character view helps make it a bit more obvious.

 

[Meneth]

What will the AI do after one month?
And actually it's not that dumb because standing on defensive terrain might be better than trying to run way and getting cought in bad terrain.


So have you told the AI that when they are in a defensive war and outnumbered, it's not their job to send their troops on the offensive? Because the CK2 AI had some very weird priorities in these cases...
And the AI embarking/disembarking without good reason or declaring wars for distant lands while under attack are also a bit weird.
For me, the most important thing is that it doesn't make these obviously weird mistakes.

After a month it'll just go back to business as usual, since we don't want it appearing braindead. It has a cooldown of 3 months before it can attempt to stand & fight again.

There's probably still cases where the AI sends their troops on the offensive in a defensive war. We hope to address more such cases after release.
For release our focus has been lately mostly on eliminating obviously broken behavior, like appearing braindead or getting stuck in a loop. But overall in my experience the AI does less silly things than in CK2.
But ofc, the circumstances where it'll do something silly will differ. So there might be some cases where it does worse.

Thanks @Meneth ! Fascinating stuff. Will we as modder able to start our own thread? For example, if I decides to make a mod that add a significant calculation to the game (relationship calculation for example) will I be able to call a graphic event (Like a short video of a grave at the dead of the last king) and split out a thread to handle that calculation so that once the "video" is over my calculation is done?

No, but significant parts of the script system is handled in parallel.

 

[Meneth]

You tighten that AI and close that gap as much as you want.

Control of ally armies should be put on the list of "basic things that should be in at launch or right after."

How many players when asked would directly tell you "I would make the trade off for more complex code and AI just for that ally army control?"

How many frustrating moments are we going to have being unable to control an ally, simply because we've decided to hedge our bets and try and make the AI worth it again.

Allies are a nearly useless feature without at least the basic amount of controls over their movements, and no amount of AI parity is going to negate that. That's great he's gonna follow me around, I just hope he's not in the one province I want to evacuate or march too. Or he's not barreling towards me when I don't need him only for both of us to choke on attrition.

It is basic.

It is beyond basic. The needed ability to simply write a letter and go "hey go siege that" or "hey form up on me we'll go whoop up on someone." I don't care how much it complicates the code, I would take strained AI growth in the future for one iota of ally control.

And it shocks me that this took so long to get in CK2, and it doubly shocks me that it isn't in CK3.

Because across the board in every launch some form of ally army control or input should be had, at launch every game.

So this is a big topic. Talked a bit to you about it in Discord, but the summary of my thoughts are:
#1 priority should be to make the AI good; not to let the player take over

Only if that fails should other measures be considered. In my experience in CK3, the actions of your allies are rarely frustrating, so I don't personally see a need to be able to control them.
However, once we throw hundreds of thousands of players at it, it's possible our opinion there will change.
If so though, it is far more likely we'll go with full control of ally armies rather than an order system. That way we'd avoid complicating the AI code and thus making the overall military AI worse and buggier, and instead just disable the AI for your allies outright.
However, that'll only be relevant if player feedback indicates that this would make for a better user experience. Our testing so far does not indicate that that's the case.

 

[Meneth]

@Meneth , how much does the GPU have to say for the performance? I have a i5 6600 CPU, 16 GB DDR3 RAM, so slightly better CPU than your old rig and sameish RAM I guess, but my current GPU is an AMD RX460, which I believe is somewhat lower than your old GPU.

GPU performance can be a bit odd. Usually the impact is pretty minimal as long as it is powerful enough to maintain 60 FPS, and then gets a bigger and bigger impact on tick speed as it gets worse than that.
Going beyond what's needed for 60 FPS does tend to speed the game up a bit, but it's less of a difference than a CPU makes.

Your CPU is a bit worse than my old CPU (15% or so weaker), and your GPU significantly worse (4x weaker), so chances are your performance will be worse than in my video.
Your GPU is worse than our recommended GPU (40% or so worse), but significantly better than our min spec GPU. So you might need to turn down some graphical settings.
My best guess is that with lower settings your framerate will be fine but less stable than mine, while your tick rate will probably be a decent amount lower but still pretty fast.

 

[Meneth]

Sorry one more question. Will big scripts affect the performance? I know you guys used the same scripting system in development, so did you run any tests to see the extend of say an immersion mod will slow the game down? Any optimization done to this? (I don't know too much about this so sorry if this is a ridiculous question)

Definitely. Script is a major performance cost.
The main optimization is the parallelization.
In some cases we also provide triggers or list builders that have more constraints but are more performant.

 

[Meneth]

Will there be an AI difficulty setting?
Many games only have a difficulty setting that gives the AI boni, but I think most players would prefer it to also be smarter (or less stupid) at higher levels.
I'd expect that there are quite a few parameters in your AI code that could make it "harder" (prioritize challenge over other goals). Also, if I remember correctly your in other games you have a limit on how much time the AI can spend giving orders to its troops (or only a limited number of stacks can be strategically controlled). Looking at the performance graphics, giving the AI more computation time at higher difficulties should not come with a huge performance penalty and players can surely understand that smarter ai = slower ai.

There's no AI difficulty setting. There's an overall difficulty setting, but that has no effect on the AI code.

You almost never see AI settings in strategy games that actually make the AI smarter at higher levels, because strategy games are usually so complicated that it's usually not feasible to make an AI that corresponds to more than "normal" difficulty. Which is why they tend to get boosts instead. Only going down in difficulty tends to make the AI less smart.
(Though sometimes games do tweak aggression and such, but that's not quite the same in my book as making the AI smarter)

Simply throwing more time at the AI is not gonna make it smarter.

 

[Meneth]

2. What would've been the disadvantages of having the rendering done on the main thread and the gamestate management on a sub-thread? Usually, rendering is done on the main thread to avoid input lag, provide a smooth UX, and avoid UI freeze (because usually the main thread owns the UI elements, and changing these elements would require any method to be invoked on the main thread, which results in what I call thread-hopping), but I'm sure you guys did discuss this, so I'd be really interested in learning why you decided against this and how you avoided thread-hopping. Not a critique, just genuine interest.

The thread that handles all the UI updating tells the render thread to talk to the GPU, so we haven't run into any issues there.

What I take away from this dev diary is, that CK3 will most likely run even faster than CK2 on most systems, is that correct?

On my machine (before upgrading) the tick rate is the same, but the frame rate significantly better and more stable.
So the main difference I expect most people (with good enough GPUs to run CK3 well) is improved frame rate.
Machines beyond 4 physical cores likely do a bit better than CK2 on the tick rate thanks to improved threading.

I have Intel Core i7-4700HQ CPU and Nvidia Geforce GT 745M laptop, can I play this game at all?

That GPU is less than half as powerful as the Intel one we currently have listed in the min spec, so your chances probably aren't great I'm afraid.
The CPU would do fine though.

 

[Meneth]

What's the horizontal axis of that graph? Seconds, minutes, in-game years?

Years.

 

[Meneth]

Will AI characters have any sense of long-term strategy?

For example, will AI characters be able to identify a rival duke whose growth threatens the balance of power in the kingdom and then act to thwart that duke?

Will an AI character be able to maintain a long-term goal such as "I want to be king" or "I hate dynasty X and want to destroy them"?

The AI code itself focuses mostly on the short term.

There's a lot of content for rivalries and such though, and content plays a huge part of how other characters act.

oh man, if possible how could a i5-4460 & and a GTX 1050 do with Crusader Kings III. i know both are above minimum, but don't know what that means exactly.

Being above minimum basically means you can run low settings at a decent frame rate.
Being above recommended basically means you can run default settings at a decent frame rate.

So since you're a bit below recommended, depending on what frame rate you're comfortable with you might need to turn down some of the settings.

 

[Meneth]

Gotcha, how many types of graphical settings are there in CK 3? if that is something you can tell/show off right now.

The standard 3D graphics settings, plus some CK-specific stuff.

 

[Meneth]

And what about AI behavior, when it in few wars? It's still wandering from one point to other, or it's pushing till win, or it can split army if it can to fight in few wars?

The AI will focus on one unless it has enough troops to handle more than one.

If i recall correctly Imperator did not have it but do you have SSAO option for people with high-end gpu ?

We've got SSAO for the character portraits.

Does the AI actively try and secure it’s Dynasty through marriages and producing heirs? And is the AI smarter about sending it’s male heirs to become bishops?

The AI marries quite a bit, yes.
Taking the vows is handled in AI script, so I'm not familiar with how that's balanced.

 

[Meneth]

So in other words, we have the usual devices to "simulate" the appearance of long-term behaviour as in CK2, without any true strategic AI behind the AI characters?

Yep.

Why complicate the code when this already works quite well?

 

[Meneth]

Is marriage AI also handled by script?
And CK2 there were these "political reasons" for quite a lot of interactions, are they still there or replaced with more concrete reasons?

Marriage AI is a mix of code and script.

"Political reasons" is no longer a thing. Acceptance reasons are much much clearer than in CK2.

 

[Meneth]

Did you experiment with double buffering game state, or at least parts of it? You can eliminate pretty much all lock waits with it if you want. Do you have statistics about how much the threads wait on locks?

Thanks for writing a performance related DD, I was a bit worried, because I have a pretty old PC (FX-8350, 16GB RAM, GTX 970), but based on what you wrote I'm pretty sure I'll be able to run it fairly smoothly.

While double buffering definitely has some advantages, it also massively complicates a lot of things, and essentially means having to rework a lot of systems from the ground up.
The time spent locking is low enough at this point that it seems unlikely for double buffering to be worth it.

We do have stats on lock waits, yes. In our automated overnights, that tends to be under 5% of the time spent by the render thread and by the gamestate thread.

considering majority people play Single player anyways I think these games should've been focused on it from the beginning.

There's focusing on single player, and then there's making multiplayer quite literally unplayable. Introducing OOSes means introducing unplayability.

 

[Meneth]

Does the one month start counting down as soon as the AI decides to start moving to a defensible location, or does it start when they reach that location? It would be strange to see them choose to stand and fight only for it to take 3 weeks to move to the defensible location and then leave again in a week.

And do you do anything to prevent the AI from moving to the same position every 3 months assuming the war keeps dragging on for some odd reason (perhaps the player is fighting 2 wars and isn't bothering with that one atm)? Or does the map just generally have enough defensible locations that it's unlikely the AI will just rubberband back to the same place every few months?

That starts counting down after they arrive at the more defensible location.

Stand & fight only kicks in if the strong enemy army is close enough to be a threat (a few provinces away at most), so the AI looping in that manner should be rare; it'll generally try to avoid getting that close in the first place, so normally it only kicks in if you're actually trying to chase them down. If it kicks in when you're not chasing them it usually just leads to them moving a province or two to be more defensible, staying there for a month, then doing something else (like try to siege something back).

Thank you so much for prioritizing fun over challenge. It sounds like every decision you made about AI is what I would have wanted, so I'm really happy about that. Question about the speed, though. Max speed seems to be a lot faster than in CK2 depending on your specs, are the other speeds similarly effected? Or will 3 speed in CK3 be the same as 3 speed in CK2?

Speed 5 is uncapped, speeds 1 through 4 have a target tick rate, so don't depend on the speed of your PC (unless your PC isn't fast enough to achieve that speed).
The target tick rates don't match exactly to CK2; they've been tweaked a bit.

Am I interpreting correctly that if I can run CK2 that I can run CK3?

Not necessarily. Especially if your GPU is weak.

Any drawbacks of the new system? Or any other erason why it hasn't been considered/adopted for CK2? Advance in technology at some point maybe?

The biggest drawback is that it takes a lot of work to apply it to an existing game.
There's also tons of tradeoffs compared to other possible systems, but going into detail on those would be out of scope.
Compared to what it replaced though, there's virtually no downside when making a new game.

So let's say the Italy AI appoints France as their "war coordinator" and while France has Italy's troops up near Normandy a civil war breaks out. Does Italy regain control of its armies in order to go win their civil war or do they just lose all control?

I believe in that case, Italy would grab its troops back.

 

[Meneth]

Because sometimes the AI does stupid stuff like wasting piety and gold when it really should be saving them to become a king

The AI does have some systems specifically for this.
All money the AI earns is divided between a few different pools:

• Short term: Vast majority of expenses
• Long term: Creating titles, buildings, a few other things
• Warchest: Savings in case of war; needs to be full in order for it to declare
• Special: Things like the Pope trying to always keep some money around in case people ask for money

This all sounds great but oh man does it bother me that the arrangement of the colors in the legend don't match up with the colors in the graph.

They do match up but they're in reverse order. It's long annoyed me too, but never quite enough that I've looked into fixing it.

What's SSAO? Not techy person asking...

Screen space ambient occlusion.
It's a lighting technique that makes things look nicer/more realistically lit.

 

[Meneth]

Why was it necessary to forbid these cross-character checks in CK2? Couldn't you just maintain a copy of the old game state, give it to each thread, and then have the threads process the updates for their characters using the information off the old state? Then you can collect together all of the results from the separate threads together in the end and compile them into the new game state.

That's what's known as a double-buffered gamestate.
While it'd work, it's also very difficult to set up, and it can make it pretty easy to read info you shouldn't have access to if you're not very careful.
It also means having to duplicate a lot of info, which can be expensive.

Does AI see the whole map? Or are they properly constrained by the fog of war?

(don't know if you read this far down the thread)

The AI simulates fog of war to some extent, so it'll stop chasing you once you're in fog of war.
It's still vaguely aware of how many troops you have and such though (which the player would know too since troop numbers aren't hidden).

I have one big question.

In CK2, the AI would manage it's demesne in a FIFO manner. Aside from it's capital, the AI would give away its oldest acquired province to get under its demesne limit. This meant that strong conquering AI rulers were constantly changing which provinces they directly controlled (constantly facing newly acquired province penalties) and in general meant that AI rulers didn't build up and develop a core demesne. It also meant that if you conquered the newly conquered territory of an AI ruler, you would deal them a serious blow by taking away their directly held demesne. Has this been changed in CK2? A better behavior would be to use LIFO and have the AI give away its newest acquired territories first. While not as optimal as having the AI evaluate each province it controls, it's superior to the old CK2 way and still keeps the performance advantages.

I guess what I'm asking is if the AI's demesne management logic has changed at all.

The AI prioritizes atm keeping the counties in its capital duchy, so that avoids most of the issue.
However, you do bring up a good point, and I'll change it to also prioritize titles acquired later over ones acquired earlier. (With the duchy thing still taking precedence over when a title was acquired)
That change won't be in on release though.

 

[Meneth]

I don't want the king trying to revoke my only title just because I am the player

The AI doesn't do things just because you're the player.

There's a few things it'll refrain from doing to avoid making it an annoying experience (like revoking your last county unless it has a really good reason), but very little it does because you're the player.

This does offer some improvement, but why not have the AI evaluate the provinces. If they just acquired one that has a wonder or one that has been built up really well, why would they give that one away before one that is poorly developed, regardless when it was acquired? I'm sure the county's development level stats are easily available to use within this calculation. It should of course keep it's capital duchy as you said, but there are things besides how long it's had a county that can be checked to determine which ones to keep for everything else. If and when we get trade mechanics involving the silk road, that should also be considered... why give up a lucrative silk road county if you have something else that isn't as valuable? You can also look into the county's culture and faith for a lesser reason to keep or get rid of a county. It can be better to give away a county that isn't your culture and faith to avoid direct revolts than one that aligns with you. Granted, in most cases, this would be your most recently acquired counties, but not always as your most recent counties could be your culture and faith already, while something you acquired earlier isn't. As I said, that should count for less than other considerations and maybe is just to use to break a tie as revolt risk isn't as important as county value.

In CK3 for certain things like title grants we've gone with decision trees rather than utility systems (what you describe).
For very constrained actions like granting titles it has the advantage of being very predictable, very debuggable, and very easy to reason about.
This means less bugs, and less time to get good results.
The main determinant is where the AI's capital is; rarely is there need for more to break a tie than that. Thanks to your suggestion earlier though, in the future acquisition date will be used to break that tie.
Replacing the decision tree with a utility function would likely result in situations with a worse outcome, and lead to a lot more programmer time having to be spent on the system (both making it, and fixing issues in it later). Adding a utility function on top of the decision tree has similar problems; less so since the decision tree still makes the same decision, but at the same time we'd be getting a lot less out of the utility function.

Thanks for the answer. Jsut to clarify, I didn't mean to imply the system should be adopted to CK2 now. The question was rather about why the CK2 system was chosen over the new CK3 one when CK2 was originally made. But I assume the best guess would be "Nobody thought about it back then"?

That predates me working for Paradox, so I don't know. But probably because threading at PDS has been a gradual evolution, and each new game has made improvements to it.

 

[Meneth]

Why is this expensive or difficult to implement? Isn't it as simple as keeping two game state slots A and B, with one marked "old" and the other marked "new" on a given day, and switching their positions daily?

Yeah uh, that's not simple to do.

 

[blackninja9939]

...huh? It's hard to get the computer to check list A on even days and list B on odd days (and remember a digit 0 or 1 for each day)?

Yeah its not that simple, the amount of time to refactor for that would be quite large.
As you've got to care about the state of the whole program and what can be accessed when, especially with a code base that has any level of tech debt or legacy in it with global state via global variables or singletons which are not tied to the gamestate itself this becomes more and more complex to do.

So its a weigh up of the time investment for that vs its gains, and as Meneth stated in the thread already waiting for the game state locks is not a huge cost for us.