[Tool Request] Event Writer/Exporter

grisamentum · Jan 26, 2018

Now, this isn't really a request I expect to be fulfilled. But it's always seemed silly to me that writing an event chain is so... clunky.

Would it be particularly hard to write a program that looks something like XMind or another organizational software, where a writer can type descriptions into a box, along with choice descriptions? And then open the box up to specify event targeting, triggers, etc, with clear branching so that they can easily work on the next event/result from each choice? And then the program could automatically write the necessary event files and localization files for the user.

It just seems like it would cut down on user errors tremendously both for PDS internally (copy/paste errors in events are very frequent) and for modders, and allow modders who need more visual aids to have help with modding.

Romulien · Jan 27, 2018

@Dayshine has made an Event viewer tool to visualize event chains for CK2, HOI4, EU4 and Stellaris. It's half the job

Groogy · Jan 28, 2018

Back in my days we couldn't even write elaborate event chains with all these fancy thinga mabobs like flags and variables. And we were better for it.

(just joking, I definitely get what you mean)

Dayshine · Jan 28, 2018

The library underlying my Event viewer and Stellaris validator could easily be used to do something like this. It's open source, and I'd happily expand it to make something like this easier for someone else to make.

It's somewhere on my todo list to do myself, I think I could expand the event viewer to do this, but there are other things to do first

SolSara · Jan 29, 2018

When I scripted the first narrative events for CK2, I used a mind map software in order to remember which event that linked to another one. Can recommend http://twinery.org/ as I've used it to visualize the links between the events and it worked pretty well. Takes a while to do though, so a real scripting tool would be better than using a separate mind map software.

Keizer Harm · Feb 21, 2018

I've actually been working on one myself. It is the no. 1 thing keeping me from finishing my own mods - every time I dive into the script I just feel silly that we have to manually count brackets in the year 2018, and I go back to that project.

My biggest hurdles (besides time management) are parsing existing events and getting a UI set up, both areas I have little experience with. I'm considering switching to C# to make the second obstacle a lot easier and perhaps I'll make a custom event serialisation for now, rather than use whatever script CK2 uses (the game I'm working from).
My end goal is to enforce scoping, meaning that it will become physically impossible to make 90% of mistakes people (like I) make in event scripting, but for the first version the pretrigger, trigger, options etc. will likely just be big textboxes. Then I can refine from there.

Dayshine · Feb 21, 2018

Keizer Harm said:
I've actually been working on one myself. It is the no. 1 thing keeping me from finishing my own mods - every time I dive into the script I just feel silly that we have to manually count brackets in the year 2018, and I go back to that project.

My biggest hurdles (besides time management) are parsing existing events and getting a UI set up, both areas I have little experience with. I'm considering switching to C# to make the second obstacle a lot easier and perhaps I'll make a custom event serialisation for now, rather than use whatever script CK2 uses (the game I'm working from).
My end goal is to enforce scoping, meaning that it will become physically impossible to make 90% of mistakes people (like I) make in event scripting, but for the first version the pretrigger, trigger, options etc. will likely just be big textboxes. Then I can refine from there.

The library I've written will parse event files, let you modify events, then write back to a file. It'll give you parsing errors to, so it points out miscounted braces

. It's in F#, so I could give you a C# friendly package fairly easily. I'm currently using it for Stellaris, but events are basically the same across all the games.

I've also done scope checking (for everything except FROM and event_target) for stellaris, which is something that could probably be extended to CK2 without a vast amount of work.

Keizer Harm · Feb 21, 2018

Dayshine said:
The library I've written will parse event files, let you modify events, then write back to a file. It'll give you parsing errors to, so it points out miscounted braces . It's in F#, so I could give you a C# friendly package fairly easily. I'm currently using it for Stellaris, but events are basically the same across all the games.

I've also done scope checking (for everything except FROM and event_target) for stellaris, which is something that could probably be extended to CK2 without a vast amount of work.

Hmm, so you have expanded it since last time we spoke? Will it parse everything, or at least put trigger, pretrigger, options, etc. each in their own strings?

Dayshine · Feb 22, 2018

Keizer Harm said:
Hmm, so you have expanded it since last time we spoke? Will it parse everything, or at least put trigger, pretrigger, options, etc. each in their own strings?

Ok, so doing this has immediately made me realise there are many things I could do to make this easier, but a proof of concept:
This is a standalone application using my library as a reference (which could be a nuget package)

Code:

class Program
{
static void Main(string[] args)
{
//Support UTF-8
Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);

//Parse event file
var parsed = CWTools.Parser.CKParser.parseEventFile("./testevent.txt");
var eventFile = CWTools.Parser.CKParser.getSuccess(parsed);

//"Process" result into nicer format
var processed = CK2Process.processEventFile(eventFile);

//Find interesting event
var myEvent = processed.Events.FirstOrDefault(x => x.ID == "test.1");

//Add is_triggered_only = true
myEvent.All = ListModule.OfSeq(myEvent.All.Append(Both.NewLeafI(new Leaf(KeyValueItem.NewKeyValueItem(Key.NewKey("is_triggered_only"), Value.NewBool(true)), Position.Empty))));

//Output
var output = processed.ToRaw;
Console.WriteLine(CKPrinter.printKeyValueList(output, 0));
}
}

Input:

Code:

namespace = test

#One event
country_event = {
id = test.1
desc = "test description"
}
#Another event
country_event = {
id = test.2
desc = "test 2 description"
}

Output:

Code:

namespace = test
#One event
country_event = {
        is_triggered_only = yes
        id = test.1
        desc = "test description"
         }
#Another event
country_event = {
        id = test.2
        desc = "test 2 description"
         }

I currently have a few preset things like `event.ID`, `event.Desc`, `event.HIdden`, but you can get to anything using `event.Tag("name")` or `event.Child("trigger")`. You then make changes, and then it can write pretty-printed output again.

It's currently very verbose in C#, but I would change it so it was just

Code:

myEvent.All.Add(new Leaf(new KeyValueItem("is_triggered_only", true)));

Keizer Harm · Feb 22, 2018

Dayshine said:
Ok, so doing this has immediately made me realise there are many things I could do to make this easier, but a proof of concept:
This is a standalone application using my library as a reference (which could be a nuget package)

Code:

class Program { static void Main(string[] args) { //Support UTF-8 Encoding.RegisterProvider(CodePagesEncodingProvider.Instance); //Parse event file var parsed = CWTools.Parser.CKParser.parseEventFile("./testevent.txt"); var eventFile = CWTools.Parser.CKParser.getSuccess(parsed); //"Process" result into nicer format var processed = CK2Process.processEventFile(eventFile); //Find interesting event var myEvent = processed.Events.FirstOrDefault(x => x.ID == "test.1"); //Add is_triggered_only = true myEvent.All = ListModule.OfSeq(myEvent.All.Append(Both.NewLeafI(new Leaf(KeyValueItem.NewKeyValueItem(Key.NewKey("is_triggered_only"), Value.NewBool(true)), Position.Empty)))); //Output var output = processed.ToRaw; Console.WriteLine(CKPrinter.printKeyValueList(output, 0)); } }

Input:

Code:

namespace = test #One event country_event = { id = test.1 desc = "test description" } #Another event country_event = { id = test.2 desc = "test 2 description" }

Output:

Code:

namespace = test #One event country_event = { is_triggered_only = yes id = test.1 desc = "test description" } #Another event country_event = { id = test.2 desc = "test 2 description" }

I currently have a few preset things like `event.ID`, `event.Desc`, `event.HIdden`, but you can get to anything using `event.Tag("name")` or `event.Child("trigger")`. You then make changes, and then it can write pretty-printed output again.

It's currently very verbose in C#, but I would change it so it was just

Code:

myEvent.All.Add(new Leaf(new KeyValueItem("is_triggered_only", true)));

That's objectively awesome. I'd love to play around with that ^^

Dayshine · Feb 22, 2018

Keizer Harm said:
That's objectively awesome. I'd love to play around with that ^^

Ok, I'll tidy up the API and put something together over the next week or two. I'll publish it as a nuget package so it should be very easy to install (no manually downloading files

) in any .NET application.

Keizer Harm · Feb 22, 2018

But, just to check; will it parse this?
If you have generalised the children, then will it handle multiple Options? And does e.g. "#small tournament in our capital province!" get attached to the description or to the is_triggered_only?

Code:

namespace = EF

character_event = {
    id = EF.1000

    picture = GFX_evt_battle
    desc = EVTDESC_EF.1000 #small tournament in our capital province!
    
    is_triggered_only = yes
    
    only_playable = yes
    
    trigger = {
        has_focus = focus_tournament
        in_command = no
        NOT = { religion = jain }
        NOT = { has_character_flag = do_not_disturb }
        capital_scope = {
            OR = {
                region = world_europe_south
                region = world_europe_west
                region = world_asia_minor
                region = world_europe_north
                region = world_europe_east
                region = world_india
                region = world_persia
                region = world_middle_east
            }
        }
    }
    option = {
        name = EVTOPTA_EF.1000 # I will attend!
        prestige = 20
        character_event = {
            id = EF.1001
            days = 30
        }
    }
    option = {
        name = EVTOPTB_EF.1000 # B-but it's scary!
        trigger = {
            trait = craven
        }
        prestige = -50
    }
}

Dayshine · Feb 22, 2018

Keizer Harm said:
But, just to check; will it parse this?
If you have generalised the children, then will it handle multiple Options? And does e.g. "#small tournament in our capital province!" get attached to the description or to the is_triggered_only?

This is the "raw" object structure from my test example above. I hope this gives you an insight into what's actually going on:

Code:

[KeyValue
   (PosKeyValue
      (Position (Ln: 1, Col: 1),KeyValueItem (Key "namespace",String "test")));
 Comment "One event";
 KeyValue
   (PosKeyValue
      (Position (Ln: 4, Col: 1),
       KeyValueItem
         (Key "country_event",
          Clause
            [KeyValue
               (PosKeyValue
                  (Position ("none", Ln: 0, Col: 0),
                   KeyValueItem (Key "is_triggered_only",Bool true)));
             KeyValue
               (PosKeyValue
                  (Position (Ln: 5, Col: 5),
                   KeyValueItem (Key "id",String "test.1")));
             KeyValue
               (PosKeyValue
                  (Position (Ln: 6, Col: 5),
                   KeyValueItem (Key "desc",QString "test description")))])));
 Comment "Another event";
 KeyValue
   (PosKeyValue
      (Position (Ln: 9, Col: 1),
       KeyValueItem
         (Key "country_event",
          Clause
            [KeyValue
               (PosKeyValue
                  (Position (Ln: 10, Col: 5),
                   KeyValueItem (Key "id",String "test.2")));
             KeyValue
               (PosKeyValue
                  (Position (Ln: 11, Col: 5),
                   KeyValueItem (Key "desc",QString "test 2 description")))])))]

I have a bunch of helper methods I use to get to different things. "node.Childs("option")" would return a list of "Clause"s with key "option". E.g. all option blocks but no "option = yes".

The Comment isn't attached to anything, so the structure would be a list ["desc", "#small...", "is_triggered_only"]. I have some other helpers I use to find all comments directly before an element which i use for the event name in my event tools

Like you say, it's completely generalised underneath. The "Event" object is just a type of "Node". "Node" has all the basic methods, "Event" has the event specific ones as shortcuts.

Edit: perhaps a better example, this is my validator for events that run every tick:

Code:

let valEventVals (event : Event) =
    let isMTTH = event.Has "mean_time_to_happen"
    let isTrig = event.Has "is_triggered_only"
    let isOnce = event.Has "fire_only_once"
    let isAlwaysNo =
        match event.Child "trigger" with
        | Some t ->
            match t.Tag "always" with
                | Some (Bool b) when b = false -> true
                | _ -> false
        | None -> false
    
    match isMTTH || isTrig || isOnce || isAlwaysNo with
        | false -> Invalid [inv ErrorCodes.EventEveryTick event]
        | true -> OK

Keizer Harm · Feb 22, 2018

Dayshine said:
This is the "raw" object structure from my test example above. I hope this gives you an insight into what's actually going on:

Code:

[KeyValue (PosKeyValue (Position (Ln: 1, Col: 1),KeyValueItem (Key "namespace",String "test"))); Comment "One event"; KeyValue (PosKeyValue (Position (Ln: 4, Col: 1), KeyValueItem (Key "country_event", Clause [KeyValue (PosKeyValue (Position ("none", Ln: 0, Col: 0), KeyValueItem (Key "is_triggered_only",Bool true))); KeyValue (PosKeyValue (Position (Ln: 5, Col: 5), KeyValueItem (Key "id",String "test.1"))); KeyValue (PosKeyValue (Position (Ln: 6, Col: 5), KeyValueItem (Key "desc",QString "test description")))]))); Comment "Another event"; KeyValue (PosKeyValue (Position (Ln: 9, Col: 1), KeyValueItem (Key "country_event", Clause [KeyValue (PosKeyValue (Position (Ln: 10, Col: 5), KeyValueItem (Key "id",String "test.2"))); KeyValue (PosKeyValue (Position (Ln: 11, Col: 5), KeyValueItem (Key "desc",QString "test 2 description")))])))]

I have a bunch of helper methods I use to get to different things. "node.Childs("option")" would return a list of "Clause"s with key "option". E.g. all option blocks but no "option = yes".

The Comment isn't attached to anything, so the structure would be a list ["desc", "#small...", "is_triggered_only"]. I have some other helpers I use to find all comments directly before an element which i use for the event name in my event tools

Like you say, it's completely generalised underneath. The "Event" object is just a type of "Node". "Node" has all the basic methods, "Event" has the event specific ones as shortcuts.

Edit: perhaps a better example, this is my validator for events that run every tick:

Code:

let valEventVals (event : Event) = let isMTTH = event.Has "mean_time_to_happen" let isTrig = event.Has "is_triggered_only" let isOnce = event.Has "fire_only_once" let isAlwaysNo = match event.Child "trigger" with | Some t -> match t.Tag "always" with | Some (Bool b) when b = false -> true | _ -> false | None -> false match isMTTH || isTrig || isOnce || isAlwaysNo with | false -> Invalid [inv ErrorCodes.EventEveryTick event] | true -> OK

That. Is. Brilliant. Dude.
I wish I had thought of that.
And it's a cakewalk to turn those trees into custom domain objects, which be put in a UI and then you can make buttons for everything and making events will be so much easier... *drools*

OK - one feature request: make a helper to check for a comment that is on the same line as an element, coming right after it. A line like: "desc = EVTDESC_EF.1000 #There is a tournament in the capital!" is common in (CK2) events, so that would be useful to be preserved.

Dayshine · Feb 24, 2018

Keizer Harm said:
That. Is. Brilliant. Dude.
I wish I had thought of that.
And it's a cakewalk to turn those trees into custom domain objects, which be put in a UI and then you can make buttons for everything and making events will be so much easier... *drools*

OK - one feature request: make a helper to check for a comment that is on the same line as an element, coming right after it. A line like: "desc = EVTDESC_EF.1000 #There is a tournament in the capital!" is common in (CK2) events, so that would be useful to be preserved.

I've now published the library on nuget as "CWTools". I have no idea how familiar you are with .NET but I can give you a full example working project if you'd like.
I added the following helper methods to try and make it a bit easier to use from C#.

Code:

public static IEnumerable<string> Comments(this IEnumerable<Child> obj);
public static string GetError<a, b>(this CharParsers.ParserResult<a, b> obj);
public static c GetResult<c, d>(this CharParsers.ParserResult<c, d> obj) where c : class;
public static IEnumerable<LeafValue> LeafValues(this IEnumerable<Child> obj);
public static IEnumerable<Leaf> Leaves(this IEnumerable<Child> obj);
public static IEnumerable<Node> Nodes(this IEnumerable<Child> obj);

The example program would now be:

Code:

using System;
using CWTools.Parser;
using System.Text;
using Microsoft.FSharp.Collections;
using System.Collections.Generic;
using System.Linq;
using CWTools.Process;
using Microsoft.FSharp.Core;
using System.IO;
using CWTools.CSharp;

namespace CWToolsCSTests
{
class Program
{
static void Main(string[] args)
{
//Support UTF-8
Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);

//Parse event file
var parsed = CWTools.Parser.CKParser.parseEventFile("./testevent.txt");

var eventFile = parsed.GetResult();

//"Process" result into nicer format
var processed = CK2Process.processEventFile(eventFile);

//Find interesting event
var myEvent = processed.Events.FirstOrDefault(x => x.ID == "test.1");

//Add is_triggered_only = true
var leaf = new Leaf("is_triggered_only", Value.NewBool(true));
myEvent.AllChildren.Add(Child.NewLeafC(leaf));
// or
// myEvent.AllChildren.Add(Leaf.Create("is_triggered_only", Value.NewBool(true)));

//Output
var output = processed.ToRaw;
Console.WriteLine(CKPrinter.printKeyValueList(output, 0));
}
}
}

Let me if you have any problems, I'd be very excited to know whether it works!
I'll make a new thread for the library at some point.

The same-line comments thing is a little tricky, I'll have a think about the best way to handle that.

Keizer Harm · Feb 25, 2018

You rock. I've started my work now

Won't get results that quickly, due to general lack of time, but I'll report once I've finished a stage ^^

ᛒᛖᚱᛋᛖᚱᚲ · Feb 26, 2018

This is something Paradox should've taken the week to develop like a decade ago.

botlord · Mar 15, 2018

@Dayshine Do you have a separate grammars file for your tool?

I have been working on a similar project, I didn't realize this already existed, but I've started by defining grammars for Stellaris files using Nearley. I'll make the GitHub for it public once it is more mature, but by defining the grammars independently from the rest of the code the grammars should be able to be ported to any other platform that supports it. There is, for example, Lark which can convert Nearley grammars into its own format for use in Python.

Dayshine · Mar 15, 2018

botlord said:
@Dayshine Do you have a separate grammars file for your tool?

I have been working on a similar project, I didn't realize this already existed, but I've started by defining grammars for Stellaris files using Nearley. I'll make the GitHub for it public once it is more mature, but by defining the grammars independently from the rest of the code the grammars should be able to be ported to any other platform that supports it. There is, for example, Lark which can convert Nearley grammars into its own format for use in Python.

No, I don't have a grammar as such, just a data structure, although I'm sure my parser is written in a way which could be turned into one quite easily. I'll PM you a link to the source code. Featurewise the parser can handle all of vanilla (.txt, .gui, .gfx) and I'm pretty confident it's complete (noone has reported an error in the parser in the last few weeks at least

)

Building a grammar is a good idea, although I'm not sure that style of generated parser will good enough for all uses. I need to be able to parse the entire mod within 10-20 seconds for my usage, which is about 3000 files for a total conversion. It's taken a fair bit of careful optimization to get it to that speed, so I'd hope a parser generator wouldn't be able to just do it

botlord · Mar 16, 2018

ᛒᛖᚱᛋᛖᚱᚲ said:
This is something Paradox should've taken the week to develop like a decade ago.

I wonder if they have an in-house tool?

Dayshine said:
No, I don't have a grammar as such, just a data structure, although I'm sure my parser is written in a way which could be turned into one quite easily. I'll PM you a link to the source code. Featurewise the parser can handle all of vanilla (.txt, .gui, .gfx) and I'm pretty confident it's complete (noone has reported an error in the parser in the last few weeks at least )

Building a grammar is a good idea, although I'm not sure that style of generated parser will good enough for all uses. I need to be able to parse the entire mod within 10-20 seconds for my usage, which is about 3000 files for a total conversion. It's taken a fair bit of careful optimization to get it to that speed, so I'd hope a parser generator wouldn't be able to just do it

Alright cool, thanks for the link! The code you sent me looks like it already has some grammar like features, I'm not familiar with F# though so I'll have to read up on it. I think a context-free grammar should be able to work on all Clausewitz datafiles. After all, Paradox probably has a grammar defined somewhere in the Clausewitz code itself. Don't know if they'd give it up though.

[Tool Request] Event Writer/Exporter

Field Marshal

Lt. General

Fossilized Platypus

Second Lieutenant

Lt. General

Swamp German

Second Lieutenant

Swamp German

Second Lieutenant

Swamp German

Second Lieutenant

Swamp German

Second Lieutenant

Swamp German

Second Lieutenant

Swamp German

First Lieutenant

Private

Second Lieutenant

Private