• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

HOI4 Dev Diary - Performance and AI

Hello! I’ve been drafted from EU4 to fight in the trenches together with the HOI4 team for a month and give some support on AI and performance. So I’m here to talk about those.

AI

I’ve been mostly devoting my time into improving garrison orders to solve some really wonky cases that could happen. Like it placing two units on Magdeburg instead of Berlin letting the capital be completely exposed. But also focused on making the garrison work a lot better when you don’t have enough troops to cover every point in the order that needs it.

First I implemented a new tool for HOI4 to let us programmers visually inspect what the unit controller is up to.

capture(95).png

Province Weights

capture(96).png

State Weights


After that came the big task, rewriting garrison order and how it assigns units. There’s been a lot of issues here and you’ve probably experienced them from time to time. Shuffling, units prioritizing the wrong states or provinces, the list goes on.

So what did I do that is new. Instead of having the old logic setting up rules for where units should go I went with a thing I hate. Mathematics. The main difference is that each state will sort of calculate how “painful” it is to remove a unit from it now. This means as each state goes through this, they will be trying to find the least painful way to perform the garrison order and will eventually settle on a sort of equilibrium.

It’s given us a lot easier way to control where units should go in a garrison and where they end up will match a lot more in style of the weights you can see on the pictures above.


A thing that we have been planning to do for a long time and finally got around to is to allow a more flexible way to control how the AI plays. We use a system we call “Ai Strategic Plans” for this. Its essentially telling Ai to act a certain way when it comes to national focuses and priorities based on scripted conditions. To put that in english I am going to hand over to @Bratyn for some examples:

Some situations in which the new Strategy Plans have been useful is to ensure ‘compatible’ AI behaviour in the case of an ahistorical Germany. A lot of the AI behaviours for the major European powers has been based on the assumption that Germany remains fascist and causes mayhem in Europe. This means that AI Italy would usually see Germany as an ally, and that AI France would either always go with the United Kingdom (if historic mode was turned on), or be so unpredictable that it was problematic for planning out and predicting alternate history scenarios.

Using strategy plans, we have made these AI nations partly behave more logical to the changing world scene, and partly ensure that the player will always have a challenging playthrough. Going democratic Germany, only to get your only enemy (Soviet Union) end up at war with both the Allies and yourself, is not a very fulfilling playthrough. Instead, a new strategy plan will now load for France upon German completion of the “Oppose Hitler” focus. This delays their pick of which political direction to head into until we have more information from the German player, essentially a ‘holding pattern’.

  • If Germany revives the Kaiserreich and chooses “Focus on the True Enemy”, France is released from the holding pattern and will go with Britain, forming a counter-reaction to rising German colonialism. Essentially, in this playthrough not much will change other than Germany being unaligned rather than fascist. The war will likely develop in a similar matter (though not on a similar timeline), with one war with the Allies, and one war with the Soviet Union.

  • If Germany revives the Kaiserreich and chooses “Expatriate the Communists”, these communists will head to France and give them a communist drift national spirit. France is released from the holding pattern and will go communist, choosing to side closer with the USSR so that both powers can stand against the new Anglo-German alliance.
  • If Germany goes democratic, France is released from this holding pattern and will choose to Revise Versailles. Which direction they pick next is up to the AI, ensuring some unpredictability. France may go Communist and start their own faction, or join the Soviet Union, ensuring a two-front war if Germany goes to war with the USSR. Alternatively, the French may go fascist. They then will go for “Woo Italy”, creating a new semi-powerful Fascist faction in south-west Europe. Similar strategy plans for Italy (and liberal application of AI strategy values…) ensure this alliance is stable. A new focus for france “Dismantle the Democracies” becomes available if Germany is democratic, and the AI has a high likelihood of taking this when the Germans are already at war with the USSR. This makes gameplay more dynamic, and ensures Germany will have ample enemies to fight against even if they elected to go democratic.
We also used AI strategy plans to teach the Chinese AI how to handle the Inflation mechanic and do things like making the PRC AI keep the peace in China when they are going for the peaceful takeover through the political struggle. All in all, to myself and @Archangel85, this has been (and will continue to be) an invaluable tool for ensuring believable and, most importantly, fun playthroughs for each ahistorical path that one can take. We hope to continue building upon this basis and make nations ‘smarter’ in dealing with alternate history situations, or just general oddities that arise in the state of the world over the course of a playthrough.

Modders I am sure are going to like this a lot, so for you guys check out the example below for how it can be set up (regular humans without modding superpowers can safely ignore ;) ).

upload_2018-2-21_16-49-6.png


upload_2018-2-21_16-49-24.png

PERFORMANCE

Late game performance have always been a tough nut to crack for us on our games. For Cornflakes we rolled up our sleeves and got elbow deep into the code to improve it the best we could.

In order to measure performance and see where it goes bad to better pinpoint what causes the slowdown I write a little script that generates this super sexy graph for us.

capture(93).png


We did tons of neat little tricks to improve our performance. AI when it micromanages does a lot less pathfinding now. The game will automatically balance workload between cores depending on how long the hourly update took for a country, The fronts system has been a lot more streamlined to make it not hog performance

And we’ve done quite A good job at it as well. Here’s the result of our hard work just this week

capture(94).png


Between finding ways to make things more efficient and utilizing the cores on the CPU better we’ve gotten an overall improvement late game. We won’t rest there though, there’s tons more that we can do to make late game an even more enjoyable experience!



To conclude this Dev Diary, we have a reminder for you. Tune in for the first day of the Three Day War later tonight, 20:00 until 22:00 CET. Since the release of Waking the Tiger is approaching we have invited some of the grandest streamers to battle it out in multiplayer and show off the expansion as well as put the build to the test.


Quill18, Alex the Rambler, Marbozir, Enter Elysium, Solar Gamer, Spiffing Brit, Drew Durnil, The Prussian Prince, Bratyn, Da9L, Shen Plays and Jay’s Gaming will all be streaming their point of view in their own channels, and we’ll be hosting a main observer stream at the Paradox Twitch channel: https://www.twitch.tv/paradoxinteractive

Next week we are going to try and do a big recap of all the features and changes and also talked about a bunch of small stuff that hasn't fit into diaries so far. See you then!


Rejected Titles:
Droogies Quest For The Missing Core
Hoi4 Dev diary: Reverse the polarity of the deflector dish
Graphs are hot!
 
Last edited by a moderator:
You should make it so that the AI stops building dozents of just Infantry divisions at once, I really hate that.

You have lots of shitty troops that are just there as a cheap way to hinder the player and that just isn't very sexy, Britain.
 
@podcat Have you guys looked into unit spam at all? Other then that this looks amazing!
 
Last edited:
Thanks all for your hard work!

PERFORMANCE
Here’s the result of our hard work just this week

Do you think there will be time for more work in the performance realm before the release of WtT?

If someone were to have a i7-7700 4.2GHz would there be a difference that a human would notice vs. the i7-6700 3.4GHz you used?
 
Nice.
Also, this will help reduce those pesky "why isn't PDX working on AI!" people that plague every single comments thread everywhere :p .

Also, Quill18 and the gang are streaming? Even better. Gives me YouTube vids to watch for a long time to come (especially since I don't have wireless at home and can't watch YT vids that often in the first place ) :p .
 
But are the AIs battle mechanics improved? That is the biggest function the AI plays is it not? @Bratyn
I find it underwhelming that all this time on a big expansion and 'The AI Dev Diary' has zero to say about battle planner shuffle,
suicide attacks etc etc
 
U should watch stream, AI in battle doesn't look to much improved...

They've also said that a lot of these changes are recent, and as such aren't in the stream playthroughs.
 
Improvement to AI Garrison and late game performance are very promising! Now I hope the AI will not leave strategic location such as Suez canal completely undefended!
 
Hey @podcat @Archangel85 @Bratyn I want to ask you again about the border of Tibet and China. Information I looked up says that Western portion of Xikang was part of Tibet and that Tibet looked almost the same as Tibet Autonomous Region looks today. Here are my sources:
https://en.wikipedia.org/wiki/Tibet_Area_(administrative_division)
https://en.wikipedia.org/wiki/Battle_of_Chamdo
https://en.wikipedia.org/wiki/Xikang
Notice that acording to the last link, the counties taken by Tibet when at war are only located in east Xikang, which makes one assume that Western Xikang AKA Chamdo was already part of Tibet. It seems like the current map in HOI4 is based on the administrative plans of Kuomintang government. I say "plans", because Tibet and Chamdo were de facto not part of China at 1936. This is a Chinese DLC and I fear that if this area is not fixed now, it may never be done.
I know it is a touchy subject, but if you are not going to change the map, at least let me know what you think about this, as this is the third time I write to you about this matter :/

Yea. I think it maybe worth looking into. Timing wise its a bit risky to mess around with the map more now in asia tho

@Groogy @podcat Question: do you need a specific tag for the Ai strategy to work, or could you make them more generic. Say if the US is in a civil war and you are commie to send them supplies and volunteers but without any need to specify that you are a specific commie country?

Also just in general how moddable is this system? :)

Actually we have a new neat thing so you can script reversed ai strategies meaning its easier to hit dynamic targets. And from focuses etc you can use scopes like FROM, ROOT etc now :)

You should make it so that the AI stops building dozents of just Infantry divisions at once, I really hate that.
We have clamped down on this a lot. production is much more solid now.

as for "did you fix X" stuff. I am gonna do a patchlog diary later with all the smaller details. A very large part of the AI work has been about making it use the new chain of command system which has required a lot of changes to front management, better leader management and such. I've rewritten a lot about template management and upgrades in the field also
 
Hey @podcat @Archangel85 @Bratyn I want to ask you again about the border of Tibet and China. Information I looked up says that Western portion of Xikang was part of Tibet and that Tibet looked almost the same as Tibet Autonomous Region looks today. Here are my sources:
https://en.wikipedia.org/wiki/Tibet_Area_(administrative_division)
https://en.wikipedia.org/wiki/Battle_of_Chamdo
https://en.wikipedia.org/wiki/Xikang
Notice that acording to the last link, the counties taken by Tibet when at war are only located in east Xikang, which makes one assume that Western Xikang AKA Chamdo was already part of Tibet. It seems like the current map in HOI4 is based on the administrative plans of Kuomintang government. I say "plans", because Tibet and Chamdo were de facto not part of China at 1936. This is a Chinese DLC and I fear that if this area is not fixed now, it may never be done.
I know it is a touchy subject, but if you are not going to change the map, at least let me know what you think about this, as this is the third time I write to you about this matter :/

I am aware of that and an argument can certainly be made that it should be part of Tibet. However, this was a case where I would have to investigate further why this particular border was chosen when the game was first released (that was before my time here), and that was more effort than this would likely be worth (it's usually a 50/50 chance between "that's how it was in HoI 3, because that was how it was in HoI 2" and "there are AI issues"). I've made a note about it and will revisit when we have a release out and are working on follow-up patch stuff.

@Groogy @podcat Question: do you need a specific tag for the Ai strategy to work, or could you make them more generic. Say if the US is in a civil war and you are commie to send them supplies and volunteers but without any need to specify that you are a specific commie country?

Also just in general how moddable is this system? :)

Plans are enabled through regular triggers, so has_government = communism is all you need. The system is pretty powerful and fully moddable, so you can for example tie different research priorities to different focus branches combined with ai_strategies to befriend certain countries.
 
Of course we've done tons of stuff and we've been several people dipping our hand into it. This was just one thing we could pick out where we could show something visually instead of just a gigantic block of text ;)

Thank the gods!

On the subject of these AI strategies, it appears everything in Europe is centered on how Germany plays it.

Has similar work been done in Asia for Japan?

I understand it is a massive endeavor to add scripts for every nations in relation to every other nation. And i wouldnt expect them all to be in now.

But can we assume in further DLCs we will get scripts added to relevant nations for reactions to the Majors updated in that DLC?

Do you think it makes sense for germany to have reaction scripts for UK/USA (particularly Democratic)? Or will the AI always have Democracies/Communists reacting to Fascist aggression?
 
I understand it is a massive endeavor to add scripts for every nations in relation to every other nation. And i wouldnt expect them all to be in now.
Yeah this is something we invented pretty late. So only a few nations have this stuff. I expect us to keep developing on it and adding more though
 
Thank the gods!

On the subject of these AI strategies, it appears everything in Europe is centered on how Germany plays it.

Has similar work been done in Asia for Japan?

I understand it is a massive endeavor to add scripts for every nations in relation to every other nation. And i wouldnt expect them all to be in now.

But can we assume in further DLCs we will get scripts added to relevant nations for reactions to the Majors updated in that DLC?

Do you think it makes sense for germany to have reaction scripts for UK/USA (particularly Democratic)? Or will the AI always have Democracies/Communists reacting to Fascist aggression?

I suspect it will be part of making the AI for new focus trees to add reaction scripts for other nations.
 
Between finding ways to make things more efficient and utilizing the cores on the CPU better we’ve gotten an overall improvement late game. We won’t rest there though, there’s tons more that we can do to make late game an even more enjoyable experience!
If I read this correctly, the mean time for a tick was reduced from 106 to 62 ms, or it will take roughly 60 percent as long to happen.

Also, the peak (90th percentile) tick duration was reduced from 233 to 118 ms, which means they will take roughly 50 percent as long.

If my understanding is correct, that sounds like a massive quality of life improvement. :)

Of course we've done tons of stuff and we've been several people dipping our hand into it. This was just one thing we could pick out where we could show something visually instead of just a gigantic block of text ;)
I think that this is a great visual example, and it should improve things both for the AI and for players using the garrison order.

I’m disappointed to not get more information about AI improvements. We’ve been seeing major problems with the AI since HOI4 shipped. Examples of issues I’d hoped to see addressed:
  • AI underproducing equipment and overproducing units, resulting in shortages.
  • AI wasting its equipment and manpower with suicidal attacks.
  • AI wasting its equipment and manpower by sending transports through sea zones controlled by the enemy.
@podcat, my question for you is whether any other major issues have been addressed in the AI. If so, I hope they can be covered here or in a later Dev Diary. If not, I’d like to know if and when we can expect to see movement in this area.
 
Questions: 1.Will you fix or change the Ribbentrop-Mołotow focus? It makes Germany give up half of poland to USRR when Poland joins axis. 2.Will you make alternate path for Germany when poland joins them? (You know no Ribbentrop-Mołotow maybe danzig for Slovakia?)
 
Of course tons of improvement for the ahistorical path and 0 for those who would like to avoid fantasy WW2.

Because our money stink.
They have put plenty of work into historical gameplay. Watch the latest stream.
 
Of course tons of improvement for the ahistorical path and 0 for those who would like to avoid fantasy WW2.

Because our money stink.

Yep, they found a way to code all those performance improvements so that if you strictly follow history, they revert back to the super-slow speeds pre-patch. Pretty sneaky of them if you ask me!

And if your money stinks, you should probably ask yourself where you've been sticking it!