Home Trending Latest threads New posts Developer diaries

ckTrim - Character Removal Tool: Discussion

Jump to latest Follow Reply
  • We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
New trimmer

I would like to note here that I will shortly release a full java library containing low and high level tools for analyzing/modifying any paradox save file (at least HOI2, VIC, EU2, EU3, CK).

As exemple of use / tutorial, the release will include:
* a trimmer which removes all useless dead characters from a CK save
* a bride finder for CK
* a save collapser that helps understand the layout/content of a savefile for any paradox game

The library will come as a jar will full javadoc documentation.
The three programs will be released with their full code, so that anybody will be able to adapt them according to their needs.
The performances are rather good.

You will have to use a JRE6 to run it.

Well, this will happen if I still can upload somewhere here (as I did for a previous bride-finder), as I don't maintain my own website.

Yves
 
Coincidentally, I'm writing a Crusader Kings Framework as we speak in .NET. I already have the Parser which reads any Paradox file (the one used for ckTrim), but I plan to create a Framework around this for CK, which will give the developer access to classes for manipulating the various game objects without having to worry about reading and writing to the save game files.

That's the plan anyway.
 
That's actually what I have done in java (not .NET). There are:

* two parsers (one slow and very generic, one fast, but requiring more memory)
* a grammatical analyzer for paradox files with callbacks included
* one set of callbacks for building the field tree
* one set of callbacks to filter out what a second set of callbacks will see (so that you don't always have to build the whole data tree)
* classes for manipulating arbitrary named fields, with the possibility of inserting new fields, deleting or modifying fields, including the possibility to write down a modified save file even though you did not analyze the whole save file (tricky that one!)
* other utility classes providing very high level means of calling another kind of callbacks (involving another grammatical analyzer) and making things very easy for the user
* classes wrapping all of this together so that the user really has very little to do. As an exemple, my trimmer is written in about 300 lines of code, everything included.

Users from non Microsoft platforms will appreciate I hope :cool:

I had already writtent almost the same thing in C++ for Windoz, but I did not release it because writting the documentation was not that easy, and I was not actually that happy with my code. This time, I have almost finished. :D

Yves
 
Just tried you ckTrim and I got an error;

Value cannot be null
Parameter name: key

See the end of this message for details on invoking
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.ArgumentNullException: Value cannot be null.
Parameter name: key
at System.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument)
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at ckParser.ckDynastyContainer._CreateContainer(ckReader& sr, ckExpression exp, String tag)
at ckParser.ckContainer.Read(ckReader& sr)
at ckParser.ckPruner.ckPruner._ReadInclude(ckExpression exp)
at ckParser.ckContainer.Read(ckReader& sr)
at ckParser.ckFile.Read(ckReader& sr)
at ckParser.ckFile.Read()
at ckParser.ckPruner.ckPruner.Read()
at ckTrim.frmTrim.cmdOpen_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.42 (RTM.050727-4200)
CodeBase: file:///C:/WINDOWS/Microsoft.NET/Framework/v2.0.50727/mscorlib.dll
----------------------------------------
ckTrimBeta
Assembly Version: 0.1.0.4
Win32 Version: 0.1.0.4
CodeBase: file:///C:/Program%20Files/ckTrim/ckTrimBeta.exe
----------------------------------------
System.Windows.Forms
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.42 (RTM.050727-4200)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.42 (RTM.050727-4200)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.42 (RTM.050727-4200)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System.Configuration
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.42 (RTM.050727-4200)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Configuration/2.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Xml
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.42 (RTM.050727-4200)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Xml/2.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
<system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.
 
I got the same error with a similar description:

Error2.jpg

See the end of this message for details on invoking
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.ArgumentNullException: Value cannot be null.
Parameter name: key
at System.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument)
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at ckParser.ckDynastyContainer._CreateContainer(ckReader& sr, ckExpression exp, String tag)
at ckParser.ckContainer.Read(ckReader& sr)
at ckParser.ckPruner.ckPruner._ReadInclude(ckExpression exp)
at ckParser.ckContainer.Read(ckReader& sr)
at ckParser.ckFile.Read(ckReader& sr)
at ckParser.ckFile.Read()
at ckParser.ckPruner.ckPruner.Read()
at ckTrim.frmTrim.cmdOpen_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.832 (QFE.050727-8300)
CodeBase: file:///C:/WINDOWS/Microsoft.NET/Framework/v2.0.50727/mscorlib.dll
----------------------------------------
ckTrimBeta
Assembly Version: 0.1.0.4
Win32 Version: 0.1.0.4
CodeBase: file:///D:/CK%204th%20Crusade/Neuer%20Ordner/ckTrimBeta.exe
----------------------------------------
System.Windows.Forms
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.832 (QFE.050727-8300)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.832 (QFE.050727-8300)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.832 (QFE.050727-8300)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System.Configuration
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.832 (QFE.050727-8300)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Configuration/2.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Xml
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.832 (QFE.050727-8300)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Xml/2.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
<system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.

~Lord Valentine~
 
Darn - there goes my 100% reliability for this release!

Could one or both of you PM me with the problem savegames attached? You'll need to zip or rar them as this will reduce the size of the file to something manageable so I can work out where the problem is.

Looking at the error message it sounds like the problem might be something to do with your dynasty files. Are you using modded dynasty files? Either way send a zipped copy of those too please?
 
Last edited:
Thanks to Lord Valentine for sending me his Dynasties.txt file. The problem exists here (tough one to spot with the eye - see if you can find it before I tell you where the problem is):

Code: 
dynasty = {
	id = ( type = 12 id = 101680 }
	name = "Tegaingl"
  	province = { 018 019 020 }
}
dynasty = {
	id = ( type = 12 id = 101681 }
	name = "Topor"
  	province = { 529 428 429 432 }
}
dynasty = {
	id = ( type = 12 id = 101682 }
	name = "Dukljanin"
  	province = {  }
}
dynasty = {
	id = ( type = 12 id = 101683 }
	name = "Kometopoulos"
  	province = {  }
}

Ok - did you find it? Basically the id in each of these dynasties is begun with a "(" (open bracket) rather than a "{" (a brace)... which confuses the parser.

Curiously this has raised it's head before because I can see where I had to debug in exactly the same place in a previous dynasty text file sent to me. So I think this is a modded dynasty file downloaded from this site somewhere.

I don't understand how CK allows this but I guess it does or your games would be crashing.

Anyway, no need for a new release.

Anyone experiencing problems as described above should open the dynasties.txt file in notepad and do a search for "id = (" . Each one you find change the "(" to a "{" and your problem will be solved.
 
So my issue was the dynasties file also - but I had thought you were refering to the 1066 dynasties file, not the DB one. :wacko:

Anyway, it works now.
 
Hey, I've got the very same problem described by Lord Valentine, but...

now that I think about it, I'm playing Deus Vult!!! It still doesn't work with DV? What's different?

edit: I've changed what you suggested and it doesn't work with DV
 
Last edited:
kokomo said:
Hey, I've got the very same problem described by Lord Valentine, but...

now that I think about it, I'm playing Deus Vult!!! It still doesn't work with DV? What's different?

edit: I've changed what you suggested and it doesn't work with DV
Click to expand...

I don't know why it doesn't work with DV. But it also doesn't have to work.

DV has its own, built in character-trimmer. It gets rid of all the useless characters, marriages, rivalries, friendships and relations at the end of each month. That is why you always have small pauze at the turn of the month.
 
Veldmaarschalk said:
DV has its own, built in character-trimmer. It gets rid of all the useless characters, marriages, rivalries, friendships and relations at the end of each month. That is why you always have small pauze at the turn of the month.
Click to expand...

Really? But take a look, figuratively speaking, at my recent game as Hungary (the one I mentioned at the main forum). I'm at 1120 and the save file is well over 30 Mbs! Is that trimmed you mean?
 
kokomo said:
Really? But take a look, figuratively speaking, at my recent game as Hungary (the one I mentioned at the main forum). I'm at 1120 and the save file is well over 30 Mbs! Is that trimmed you mean?
Click to expand...

Yes, the game gets trimmed. I don't make that up

I have just recently cleared my save-game folder. But the biggest save-game I have is from 1150 and it is 28MB.

Remember that a save-game from DV contains much more information then that from vanilla-CK
 
Fine, the game gets trimmed. But still, the ingame trimming doesn't seem to get rid of all the characters this old thing did. If I remember correctly, it first removed all the dead childless characters, then the ones who becam dead childless characters after the first trimming etc.
 
What makes you think DV doesn't remove these characters?

Using DynasticGlory I found that entire dynasties had disappeared from the savegame, shortly after the last character of this dynasty died.

But as long as even one character with a family link survives, they remain in the game.
 
Yes, that's the thing. I admit that I haven't tried the old trimmer in a while, but the way I remember it, it also removed the dead childless characters with living relatives. For instance, all the dead infants were removed. That doesn't seem to happen with the built-in trimmer.
 
I don't know how effective DV is at trimming. ckTrim did get files down to about half (looking at my old savegames up to the end of the 12th century, they came to about 15mb after trimming).

As was said earlier, the game does now have more relationships which will account for an increase in size, although I'd be surprised if these alone caused the save game to double in size compared to CK.

One way to find out I guess is to redo ckTrim and see if it finds any new characters to trim, but if it doesn't, it'd be wasted effort and there doesn't seem to be a massive call for an upgraded ckTrim otherwise.
 
excellent. spent a few hours trying to figure out how to work some of the other options out there. this is the only one that actually worked.

thanks.
 
You must log in or register to reply here.
Top Bottom
Back