Home Trending Latest threads New posts Developer diaries

Paradox Savegame Toolkit

Jump to latest Follow Reply
  • We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
U

unmerged(144889)

Private
2 Badges
Jun 28, 2009
11
0
  • Europa Universalis III
  • Europa Universalis: Rome
Hope I'm not getting banned for multiposting, since I'm gonna post this in the EU3 forum as well (and already did in the useful utility library thread).

Paradox Savegame Toolkit

written in C++, open source. Obviously not just for savegames but all files using that common paradox text file format.

includes an easy to use API for analyzing/modifying savegames for different paradox games. Because of the bundled tools I compiled it as DLL, but feel free to include it directly in your program (as long as it's GPL'ed obv.). Should be well suited for both console and GUI apps (optional status/progress callback).

Tools included:

romestrip - for stripping rome savegames
eu3strip - for stripping eu3 savegames
eu3custproc - example app reducing citysize logarithmically so that 1Mio cities will have about 500k while 1k almost remain the same.

I had written the base class and the rome stuff over a year ago, but never got around to release it. The eu3 stuff I did add a couple weeks ago, after my game (early 1900s) was getting slower and slower, while the filesize was getting ridiculously big (and the ledger started to crash the game). Tried a stripper (female) from this forum, but it couldn't handle the 50+ MB files, so I pulled out my old stuff and wrote a little tool for trimming eu3 savegames. My 1900s savegame was reduced by 66% (I'm also purging indentation, savegames are still loaded right, don't now if that works for rome too, probably yes, but didn't try).

sorry for the quick writeup, don't have much time right now.

download: see post in useful utility library thread http://forum.paradoxplaza.com/forum/showthread.php?t=282925&page=2
 
Noirin said:
Could you please put it also in another place?
If I try to get it from there I get this error: "you do not have permission to access this page".
Click to expand...

Having only Rome, but not EU III I have the same prob...
 
Hi guys, sorry for taking so long to respond. Just after posting this stuff my computer blew up (actually quite literally) so that took me a while. After that I wanted to add example GUI applications for all tools included before releasing the second version (which I will also post in this thread).
So far the eu3 and the API part of my toolkit doesn't seem to gather a lot of interest, there is basically only interest in the rome stripper part. I decided against posting it in here as-is, because I'm pretty sure it isn't gonna do you guys any good, since it was done for vanilla Rome. As I understand it there have been a lot of additions, especially in the character part of the game with Vae Victis (which is also the part contributing most to the gigantic file sizes). I don't have Vae Victis yet (I don't suppose there is a demo version?) However if one of you could upload one of your savegames (a late game, as big as possible), that should enable me to fix most of the issues (probably all of them) with added references to characters and so on.
So if you'd bear with me a little longer, you will get a rome stripper, that can also handle Vae Victis savegames. If any of you only has vanilla rome, or would like to try the stripper as-is (it most certainly WILL fail, but you might be able to use the strip identation mode and what else there was, but not the irrelevant character purge, which contributed the most memory gain), just drop me a line and I'll send it to you. Like I said, next release will also contain GUI apps for all the tools (almost done). And please, if any of you would be so kind to zip a VV savegame and send it to me. Thanks in advance!

Marcus Scipio:

it's not a program, it's a collection of tools (which themselves are more or less meant as demo applications for the underlying "framework" for writing console/GUI paradox savegame processing applications)
 
Doh, I just deleted one of my larger files, since I wanted to start over with Sparta during 474 and work my way up slowly.

Am I correct in assuming the main things you strip out are extraneous entries? I was looking through the savegame file and found an entire section of previous_war, which I assume is only referenced by the character's history data for the battles, is that something which you're targetting?

I can't seem to view the previous_war data within the game via the ledger. Would this just be a candidate to wipe out? Make it so only character history data is available for current running wars?

Only save game file I could find is 9M.
 
Do you have a list of API hooks? My C++ is a bit rusty, but I can work in something managed (Java, .NET) and would love to tinker (I also have a save game parser and interface for CK (VB or C#, can't remember which I used...), but never posted it since it was the first component of a never completed family tree generator...)
 
Speaking of which, is there a published format via the dev diaries for the savegame format? Does the spacing matter? It would be far easier to change the indentation to be more "sane". It's pretty easy to slurp the entire file into perl or python right now and parse it up. I just don't know what's safe to delete.
 
peppermints said:
Hit's not a program, it's a collection of tools (which themselves are more or less meant as demo applications for the underlying "framework" for writing console/GUI paradox savegame processing applications)
Click to expand...

Okay, but what does it do? I'm sorry, I'm a total n00b. What you're saying is, whit all due respect, like Chinese to me.

Latin or Greek, yeeah, I would have got that, but what are you saying?

It reduces a safe file in size by removing Data you don't need? :confused:
 
Marcus Scipio said:
Okay, but what does it do? I'm sorry, I'm a total n00b. What you're saying is, whit all due respect, like Chinese to me.

Latin or Greek, yeeah, I would have got that, but what are you saying?

It reduces a safe file in size by removing Data you don't need? :confused:
Click to expand...

If you start a game in 474 AVC, the savegame file is about 3.2MB. By 499 AVC it's up to 12MB, which makes save/load much longer. You can set the autosave interval to be longer to alleviate the slowdown, but there's a lot of extraenous data within the savefile.

From just browsing the file I can think of a lot of candidates for removal. The history information for a lot of characters is quite spammy. One refugee ended up getting Cupbearer/Huntsman/Seneschal/Physican 5 times due to 4 Civil Wars and finally 1 Annexation. The ledger information and every single battle being recorded, long after the characters who fought them are dead.

From a character history point of view, do the preceding seed values matter or only the last seed? If so then a lot of the add/remove traits and childhood events can be wiped leaving only the last entry.

Cival Wars generate a ton of character history spam. Might as well remove all reference to granted titles that are later stripped. The AI tends to dump these on unloyal subjects, then the subject defects to your country anyways with baggage in tow.

As mentioned before the previous_war entries have a ton of data in them. Is there any reason to keep them around afterwards? I can't find a way to reference that data within the game ledger. Am I missing something? I know the data is partially referenced by the character history, but to be honest I'd only care about knowing their current war statistics in order to compact down the savefile. Another possibility is to strip off all previous_war history entries of leader ID's who are dead. Without being able to reference the data from within the game, it makes most of the data effectively useless.

Chopping off all references to previous wars for a 25 year game for me will remove about 8-9k lines from a 590k file, but will also reduce the amount of ID mapping the game likely has to do in order to load the file.
 
Last edited:
With a 14MB file, I hacked up a quick perl script wipe all entries for

war={ name="" history={ } } }

which clears out all the battle data on barbarian fights. That chopped off almost a MB of size. Each fight within the history section has to map to 2 possible ID tags, so I'm guessing the game needs to hash these values in some way so they're referenced on demand.

I didn't time my pre/post game load, I probably should've.

I also chopped all previous_war={ } sections, which depending upon how active you and the AI are, chopped off another 1MB of space. Since the war itself has an action={ } stamp with a YY/mm/dd, it should be configurable to leave in years within the last 10 years.

I haven't parsed all the character history data, but I believe it's safe to assume anything wthin character={ } which is defined as purely numeric ID. A lot of Event ID's could be stripped out, if you only care about the "big stuff", namely when they gain titled, switch countries, get married, etc... If they're dead, maybe chop that history.

Even for Kingdoms and Tribes each character entry has their party status. An odditiy is when you sent that character as a diplomat to another country you'll see them gain a faction icon. I haven't tried deleting them to see if they're optional or not.

Another oddity while doing save/load of new game files, am I correct in assuming all romans are defined via history/character, otherwise their names won't map correctly to the common/cultures.txt localization/culture.csv? I tried playing with switching a barbarian country to Latin culture, and none of their names registered in common/families. They all ended up being "IUS/IA", instead of "II". That seems like something that can only be fixed within the code engine, and not via user interface hacking. I did notice someone posted a mod of adding more roman women.

The modding manual thread seems incomplete. Is it assumed most of this information can be found in EU3 (which I've never played).
 
The stripper included in the kit already has options (as far as i can remember) for purging historic wars and removing the indentation. I'd expect that part to also work for Vae Victis. The character purging had among others, options for removing all childless + dead characters (which was about half of the characters in vanilla, effectly cutting the file size in half). That part is probably not gonna work as-is with Vae Victis.
Since I haven't received any Vae Victis savegames as of yet, I will upload all the current stuff and put it up here, as soon as I get home.
As for the api part. Well, currently the dll exports all the classes and you can define callback functions for handling processing/status updates. I'm afraid you'll not be able to use it with .NET as-is. I could write a wrapper to provide managed classes .NET can work with or an alternative API using no classes at all.
 
You must log in or register to reply here.
Top Bottom
Back