• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Hoi4 Dev Diary - Telemetry and data

Hello there, the time has come to do a fun dive into what you've been playing the game as a community. So over this diary we are going to look into some of the trends and statistics we seen from our telemetry and see how we use it to gain insights in current player trends when working on HOI4. The wonderful data team have been working to get us all the information we wanted and I hope you enjoy seeing some of their great work. As a forward point, as with any data collection we do not store any personal information or anything that can used to identify players in accordance with GDPR.

So first of we will start with something fun, can you guess what this graph is telling us.
1645107087765.png


If you guessed this was us spotting the continuous naval production focus exploit being discovered you get a gold star, although it seems some of you are still very much into using the focus.

With telemetry we can work to understand our players by seeing how you play and this lets us form an experience players will enjoy. we see this in many ways, for example we can see by player difficulty settings players really shy away from the harder difficulties.
1645107138178.png


We can also see how players are reacting to new content, here we can see the soviet union and Poland grow and maintain solid new highs in terms of sessions played. Germany still remains you favourite major to play, and for those of you that like playing minors the PRC seems to be the country to go for.
1645107317495.png


Now lets get into some more specific game related data, player play behavior's are something that always interesting to know. You might think you know what most players are doing from discussions on various discussion platforms, streams and videos, however what's even better is knowing for sure what the meta and average player does.

For example many people really need to check their infantry equipment stockpile late game
1645106739776.png

Seriously players really enjoy making infantry equipment 3
1645106760437.png
Alright that's allot of graphs, and we know graphs are not how we get views, so lets take tried and trusted top 5's for spin.

We will start with rapid fire round of nukes so.

Top 5 target nations:
  1. Germany
  2. Soviet union
  3. Japan
  4. USA
  5. England
Top 10 Target provinces:
  1. Tokyo
  2. London
  3. Berlin
  4. Hiroshima
  5. Moscow
  6. Nagasaki
  7. Washington
  8. Osaka
  9. Nagoya
  10. Paris
Top 5 countries nuking:
  1. Germany
  2. Soviet union
  3. USA
  4. England
  5. Italy
As we can see players are really not very imaginative with their nuke targets or they share a strange wish to nuke all the same places. Please also let me know why you really like nuking Paris in the comments below.

We can now take a trip over to something newer, its that new shiny toy the tank designer. I must say if there was a meta with tank design someone forgot to tell players because the only consistent design trend has been making cheap flame tanks.

So here we have your favourite modules for each category over the last month
MODULE_TYPEMODULE_NAME
ARMORtank_welded_armor
CHASSIShv2
ENGINEtank_gasoline_engine
GUNtank_small_cannon_2
SUSPENSIONtank_christie_suspension
TURRETtank_medium_three_man_tank_turret
extra_moduleadditional_machine_guns
extra_modulewet_ammo_storage
extra_moduletank_radio_1
extra_modulesloped_armor

How about your favorite Tank role designs, featuring some monster MBT design that players love.
ROLECHASSISGUNTURRETSUSPENSIONARMORENGINEMODULE1MODULE2MODULE3MODULE4ENGINE_UPGRADEARMOR_UPGRADE
mmbt0hc3mbttorcasdierad3smokeasyslop2020
mlt2csl2mchrrivgasrad1---------32
mlt2sc2l2mchrwelgasrad1---------102
mmt2mh1m3mbogrivdierad2tsctscmg64
mlt2ac1l2mchrcasgasrad1---------32
mlt2csl3mchrrivgasmgmgmgmg90
mlt2sc2l2mchrrivgasrad1---------32
mlt2csl3mchrwelgasrad1---------102
mlt2ac1l2mchrwelgasrad1---------32
mlt2fltl2mchrrivgasrad1---------32

You guys also have some erm "interesting" collective opinions when it comes to amphibious tanks.
ROLECHASSISGUNTURRETSUSPENSIONARMORENGINEMODULE1MODULE2MODULE3MODULE4ENGINE_UPGRADEARMOR_UPGRADE
ammt3mc2m2mchrcasturrad3amphweteasy200
ammt2mc2m3mintwelgasrad3mgmgamph60
ammt3mc2m3mtorcasturmgslopamphrad3110
ammt3mc2m3mchrwelgasrad3slopamphmg99
amhv3hv3h3mchrcasturrad3amphslopsque2020
amhv3hc2h3mintwelgasrad3slopsmokamph88
ammt3hv2m3mtorwelgasrad3slopsmokamph56
ammt3mh2m3mchrcasturrad3easywetamph200
ammt2mc2m3mbogrivgasrad3mgmgamph60
ammt3mc2m2mtorweldieweteasyamphrad31510

We have also seen that about 10% of you and only about 4% of total designs are made with auto design, this is something that reinforces our desire to improve this element of the designers.
1645106778449.png


Alright lets get into some more gritty details. first off we have been seeing how player designs have been changing since release in response to changes in patches and the design meta developing.
1645106789765.png

1645106795449.png

1645106800452.png

Next up lets see how you've been using those handy preferred tactics, it neck and neck over what players think is the best one for the army.
1645106830179.png

But you have a clear favourite for generals.
1645106844393.png

But what about all those new focus trees, well lets take a look at how you traverse focus trees. here we look at what exclusive choices you are making when playing with the new focus trees. As a community players will always trend toward the path that lets them expand and be independent so most of the breakdowns are as expected. however its also very useful to know what specific choices players make within these paths.

sov_focus_selection.jpeg

pol_focus_selection.jpeg

lit_focus_selection.jpeg

lat_focus_selection.jpeg

est_focus_selection.jpeg

axis_allies_baltic_focus.jpeg

sov_industry_focus.jpeg

Construction now, when to switch to MIL's from CIV's is always an topic for debate. as we can see though the change happens on average quite early with the first year averaging at 13.17 CIV's built. This is consistent over most majors outside SOV, with many being much more extreme than the average.
1645106861728.png

Additionally you always need more railways and infrastructure.
1645106873810.png

Finally I'll bring us to Division meta. this might be surprising but meta divisions while definitely popular, are not as ubiquitous as online discussion would have you believe.
1645106884415.png
As you can see we have the usual suspects of the 10-0's, 7-2's and 14-4's but they are nowhere near dominant in the average game.
The situation becomes even less unified once we take support companies into account
1645106896799.png


That wraps it up for this Dev Diary, If you have anything specific you're interested in from this brief overview let me know and maybe we can do a deep dive in future.
 
  • 60
  • 52Like
  • 8Love
  • 3Haha
  • 2
Reactions:
That would mean either having multiple AIs, which would need separate support and development, or force the AI to take decisions that itself would consider bad. Neither sound like a good scenario.
Lots of games have different AI behaviour for different difficulty settings. Would love to see this for HoI4, too.
 
  • 1
Reactions:
While I doubt they are going to introduce different AI difficulties anytime soon, I would love to option to be able to tweak the modifiers the harder levels have. The number one reason I do not play the hardest level more is because it crimps the player PP so much. Often, I want a harder game, but still have PP to play with all the fun tools the game has. Missing out on all that is removing to much fun.

I guess such an option would throw the stats off some, but maybe they could just add a category for Custom Play Level.
Even just being able to separate combat and industrial penalties would be really welcome.
 
  • 2
Reactions:
The infantry equipment is also good for lend leasing.

In my current USA game I have tons of infantry equipment because I have enough divisions and don't want to mess with templates anymore.

Just updating tanks gets a big cumbersome after a while.

Also I must be an outlier because I never use medium or small cannons.
 
  • 2Like
Reactions:
It's perfectly normal and helps them to understand how the game is being used and hopefully to focus on things that help the player base (maybe even better tutorials!)
Other than Paradox games, I usually play games from smaller companies. So, while Steam probably collects data on those games, I don't think those companies have enough staff to do anything with the data. Collecting data like this might be normal for the big publishers, but (as far as I know) this is the only game company that I deal with that does it.
The data is usually aggregated so it's not like people are sitting there looking at your exact play style and judging you.
I think my concern is mostly the potential for more intrusive use of the data. However, based on the responses I'm seeing here, I think I'm probably just being too paranoid on this subject. I'll just have to live with it. Thanks for commenting, I appreciate your opinion and input.
 
  • 1Like
Reactions:
I disagree with the interpretation that this is detailed monitoring.
I'm thinking it's detailed because of the amount of play data being monitored, not the amount of personal information. I'm OK with monitoring how many players play each country, but when the data gets into details like how many players are putting flamethrowers (or whatever else) on their tanks; for some reason that seems to be too much for me.

It's like a journalist aboard helicopter flying over a crowd. They don't focus on any one face, they only make observations about what the crowd as a whole is up to.
As I said in the post above, I think my concern is more the potential here. To use the helicopter analogy on privacy: while the journalist is only observing the crowd right now, they do still have the ability to either hover lower or use magnification to focus in on individual faces. This also applies to the level of play data collected. As an example: someone playing Germany sp might jokingly use names for their fleets or army groups that most people who know the seriousness of the regime might find offensive, or might even violate German laws. In some future setting if Paradox finds this in the data collected, are they going to send them a warning? Or even threaten to revoke their game key if they do it again? (I don't do this by the way, but I can see the potential for it - I used to be young - a long time ago).

This DD is making me think about those kinds of issues. However, I can see from your response and others that I'm on the more paranoid side of the spectrum - which doesn't surprise me. Thanks for replying and sharing your viewpoint.
 
For example many people really need to check their infantry equipment stockpile late game

Seriously players really enjoy making infantry equipment 3
Yep, because manually calculating the optimal factory allocation is a huge pain in the ass. I always end up with an obscene excess of guns, or a devastating shortage.

Also- this isn't just a player problem. I can always see through the Intel tab that even minor AI nations often have 500'000 guns stockpiled as well.

Players have made online tools to assist with planning production but they're usually:
  • Broken
  • Out of date
  • Confusing to use
  • Don't actually tell me what I want to know
  • Hidden on page five of google
  • Hidden in an obscure reddit comment
  • Hidden in a discord server
  • Hidden on the forums
  • Take too much time to bother with
HOI4 really needs an AI Assistant for factory allocation.
 
  • 6Like
  • 2
Reactions:
Re:Templates

I'd really like to see that data parsed by session-hours.

My France runs usually included mass-producing the default template before turning like 10-20% of my army into space marines, and though I actually never played SOV before NSB, I figure a lot of people might've employ similar tactics, and carried them over.

With SOV making up so much of the data, I wonder how much of this is people just spamming Red Army units to reconfigure later?
 
  • 1
  • 1Like
Reactions:
This topic is usually at the forefront when we talk data, and it is understandably something that people tend to be suspicious of - often for good reason.
It looks like I'm that suspicious person today. I appreciate you taking the time to respond - I know that you are very busy today.

  • Telemetry data is aggregated - while we can of course look at certain subsets, those subsets are limited by what data we collect
  • Telemetry data is anonymized - we collect no personally identifiable information on our users*
Are the limits on aggregation and anonymization set by Paradox or by Steam?

  • We use this data to inform what we do. This can range from what we focus an expansion on, to detailed tank module balance. If something sticks out in the aggregated data, there is usually a reason.
I think you can see my concerns from what I have posted above and at this point I'm not as concerned as I was when I made the first post today. I definitely appreciate your transparency on this. I have tried to choose my words very carefully because I don't want you to get criticized by your supervisor or peers for being too transparent. I think that you and the HOI4 team are doing a great job with the game, so if someone uses my posts today to criticize you for being too transparent, then I am sorry about that.

  • We do not use this data to market things to you in the same way that, for example, [your legally distinct social media website] might suddenly display a bunch of adverts relating to [your legally distinct brand of hard boiled eggs] simply because you mentioned how much you wanted an egg within speaking distance of [your legally distinct brand of mobile telephone]. (Because of the point above, we could not do this even if we wanted to. Which we do not. It's a line I have no intention of crossing.)
Two final points and then I won't post any more on this issue:
1 - I have no doubt that you personally won't do any of the things I have described in my posts today; however, when you move on to a bigger and better job, your replacement will likely have a different way of doing things.
2 - While HOI4 does not do this, the launcher does. I cannot go into HOI4 or EU4 without the launcher informing me that PDX is on Facebook and that I can buy the subscription service. And the last time I loaded HOI4 on my computer, before I got to the launcher, I was stopped at a screen listing the DLCs I did not own with links to purchase them. I already know all of that and telling me something that I already know doesn't improve my play experience - it makes me think that Paradox thinks I'm stupid.

I have been a customer from day one with EU from Strategy First. Paradox has a uniquely excellent reputation in the industry because it has always been transparent, had excellent communication with its customers, and produced a high-quality product. Paradox's reputation is not because of glitzy advertising or fun events or colorful graphics or constantly pushing advertising on its customers. When I see what Paradox is doing with the launcher it makes me think that at least a portion of the corporation sees its reputation not as something to nurture and maintain, but as something to strip mine for profit. I'm not entirely comfortable with a corporation that greedy collecting data this detailed on me.
 
  • 1Like
Reactions:
Lots of games have different AI behaviour for different difficulty settings. Would love to see this for HoI4, too.
How many of those are grand strategy games? Some game genres are easier to create different AI behaviour for.
 
  • 4Like
Reactions:
Are the limits on aggregation and anonymization set by Paradox or by Steam?
By the EU's GDPR (General Data Protection Regulation) law. It's why I'm comfortable with the metaphorical helicopter journalist - if the journalist ever exercises their ability to focus on an individual face, they are legally liable.
As I said in the post above, I think my concern is more the potential here. To use the helicopter analogy on privacy: while the journalist is only observing the crowd right now, they do still have the ability to either hover lower or use magnification to focus in on individual faces.
 
  • 1
Reactions:
Cheers for the DD C0RAX, the extra info, and the reassurance from Arheo that you're not building up a statistical picture of me to target naval-themed DLC ads in my direction :)

All very interesting stuff, and I have no doubt useful for assessing things.

In the off chance it's useful info, my interest in never going above normal difficulty settings is I don't really enjoy "grind 'em ups" against AI with statistical buffs - so would much rather play normal and then self-handicap (either by playing a major in a role-playing way and generally letting the battleplanner do most of the work, or by playing a minor that has to work hard to make an impact) to get my preferred level of difficulty, rather than get it through production buffs - but in saying that, I'm not suggesting the higher difficulty settings aren't a good thing, and given the player base, even with those relatively small percentages, that's still rather a lot of people enjoying them :)

For a naval-themed pic, I know it's a bit of a stretch, but here's a pic of a very early (1932) echo-sounder recorder, which records information from a ship's echo-sounder - ie, its a telemetry device :)


Double-spiral echo-sounder recorder.jpg
 
  • 5Like
Reactions:
Honestly not surprised the civil war paths for Russia are hardly played, the civil wars make you lose a lot of time you could be using to build an industry and Stalin seems to get the most flavor anyway if you get that far in the focus tree.
 
The second and third most used tank design are for the standard german and soviet light tanks already in production in 1936
Your observation raises a question. How is a game counted in the first place? If it is simply a function of each new game counts as a full game? If so, then your observation that tanks in production at the start of the game are the second and third most used tank design may indicate that there are many games ended early and skews the stats.

Of course, who knows for sure. It is just interesting what such a large indicator of strange behavior could mean.
 
  • 2Like
  • 2
Reactions:
Your observation raises a question. How is a game counted in the first place? If it is simply a function of each new game counts as a full game? If so, then your observation that tanks in production at the start of the game are the second and third most used tank design may indicate that there are many games ended early and skews the stats.

Of course, who knows for sure. It is just interesting what such a large indicator of strange behavior could mean.

On the nations played graph it splits them between sessions played for > 5hrs and sessions > 15hrs. So presumably they can limit it to sessions > 2 hrs or so to cut out all the sessions where people just loaded the country up to see what they are producing. No idea if they did or not tho.
 
  • 3
  • 1Like
  • 1
Reactions:
How many of those are grand strategy games? Some game genres are easier to create different AI behaviour for.
I believe Stellaris has different AI behaviour for different difficulty levels, also the Expert AI mod for HoI4 allows you to toggle AI behaviour settings. For example how much they min-max their divisions.
 
  • 2Like
  • 1
Reactions: