Documentation for the classes and functions?

[Ellye]

I'm a bit lost here; I was reading the modding help files, and I got the basic of it.

But suppose I want to override the behaviour of how a Colonist look for a job; in order to do that, I need to know the name of the Colonist class, the name of its properties, the name and parameters of its functions - and also the same for all the stuff related to buildings and jobs and such.

There are bits and pieces of those around the examples in the manual and the premade code, but isn't there somewhere with the whole thing? No need for the actual code itself, but at least the class signatures so to speak - just the name of the class, the name of its functions, parameter types, the name and type of its properties, etc.

Is there a console so I can at least easily iterate through the objects to see if I can find the names?

Thanks!

 

[Whitecold]

Where are the modding help files? I can't find anything but some packaged files in SteamApps/common/SurvivingMars

 

[Orrie_]

Had the same roadblock when I tried out some modding.

Extracting the HPK files is possible, but decrypting/decoding the lua files is another story.

 

[Jammi]

Had the same roadblock when I tried out some modding.

Extracting the HPK files is possible, but decrypting/decoding the lua files is another story.

Yes. Unfortunately, I wasn't able to decompile the files, however, I'm not very used to that. But with the files as a basis there would be nice opportunities... I (we) need it to do real modding.

 

[cpeosphoros]

Where are the modding help files? I can't find anything but some packaged files in SteamApps/common/SurvivingMars

Yes, I'd like to know that, too.

 

[Gyoudai Wulfson]

Looking for the same thing...I'd like to poke around with the Colonist desires, but I need to know how to address what I'm looking for. API reference, plz

 

[Ellye]

Where are the modding help files? I can't find anything but some packaged files in SteamApps/common/SurvivingMars

/steamapps/common/Surviving Mars/ModTools/Docs/index.md.html

 

[cpeosphoros]

/steamapps/common/Surviving Mars/ModTools/Docs/index.md.html

Not available on Linux...

 

[Whitecold]

Neither on Mac...
Do I have to boot into ugly Windows and install the game there again?

 

[cpeosphoros]

Neither on Mac...
Do I have to boot into ugly Windows and install the game there again?

I'm downloading it on Wine Steam just now. If it doesn't work, I'm seriously considering asking for a refund on Steam. I bought this specifically with the modding capacities on mind.

 

[cpeosphoros]

I'm downloading it on Wine Steam just now. If it doesn't work, I'm seriously considering asking for a refund on Steam. I bought this specifically with the modding capacities on mind.

Amazingly, the game runs on Wine. However, it is unplayable - there are severe glitches on map rendering - and, most important to this thread, Mod Editor doesn't run. When I click on that option it flashes (open/close) a "Ged" window and then replicates the behavior of the "Easy Start Option", instead of opening the Mod Editor application.

Running Ged.exe from terminal results in a stream of errors and a black "Ged" window.

So, no modding on Linux, at least not the easy way. It would probably be possible to mod in Linux anyway just by directly editing the involved files, but that would also be kind of pointless, since Steam Workshop uploading is handled in-game by the Mod Editor anyways.

I will wait until tomorrow for an answer from the devs about this. If there is no answer, or if they answer there are no plans about porting the Editor to Linux, I will go for a refund. Very sorry about that, since this seems to be a game with so much potential.

 

[mk-fg]

I've poked at lua files a bit, and despite weird LZ4 magic, it seem to be plain/uncompressed bytecode starting from 0x12, so possible to decode into something that'd still be pain to read through.
It's Lua 5.3, but apparently not the lua.org or LuaJIT one, with slightly different headers and layout.

There are documented I/O functions, so it should also be possible to enumerate stuff from lua itself, or dump same bytecode for specific things for easier inspection.
Though introspection in Lua is very basic afaik, so here's hoping for some generated API docs too - just a list of function signatures to override/tweak (as done in "Shadowed Solar Panels" example) would be very useful.

So, no modding on Linux, at least not the easy way. It would probably be possible to mod in Linux anyway just by directly editing the involved files, but that would also be kind of pointless, since Steam Workshop uploading is handled in-game by the Mod Editor anyways.

Note that GoG version also says "modding uploads/downloads coming soon", so probably safe to assume that there will be changes there in the next patches.

 

[StrategicWarlord]

I'm a bit lost here; I was reading the modding help files, and I got the basic of it.

But suppose I want to override the behaviour of how a Colonist look for a job; in order to do that, I need to know the name of the Colonist class, the name of its properties, the name and parameters of its functions - and also the same for all the stuff related to buildings and jobs and such.

There are bits and pieces of those around the examples in the manual and the premade code, but isn't there somewhere with the whole thing? No need for the actual code itself, but at least the class signatures so to speak - just the name of the class, the name of its functions, parameter types, the name and type of its properties, etc.

Is there a console so I can at least easily iterate through the objects to see if I can find the names?

Thanks!

+1 for the console and the console commands. We really need this.

 

[Waywocket]

I've poked at lua files a bit, and despite weird LZ4 magic, it seem to be plain/uncompressed bytecode starting from 0x12

I'm not sure about that. Here's a hex dump of the beginning of _GameConst.lua:

00000000  1b 4c 75 61 53 00 19 93  0d 0a 1a 0a 04 04 08 78  |.LuaS..........x|
00000010  56 00 00 00 00 00 00 00  00 00 00 00 28 77 40 01  |V...........(w@.|
00000020  19 40 4d 61 72 73 2f 4c  75 61 2f 5f 47 61 6d 65  |.@Mars/Lua/_Game|
00000030  43 6f 6e 73 74 2e 6c 75  61 00 00 00 00 00 00 00  |Const.lua.......|
00000040  00 00 02 13 ab 09 00 00  06 00 40 00 46 00 40 00  |..........@.F.@.|
00000050  47 80 c0 00 47 c0 c0 00  0a 40 80 80 06 00 40 00  |G...G....@....@.|
00000060  46 00 40 00 47 80 c0 00  47 c0 c0 00 86 00 40 00  |F.@.G...G.....@.|
00000070  87 80 40 01 87 40 41 01  52 80 80 00 0a 40 00 82  |..@..@A.R....@..|
00000080  06 00 40 00 46 00 40 00  47 80 c0 00 47 40 c1 00  |..@.F.@.G...G@..|
00000090  0a 40 00 83 06 00 40 00  0a 00 c2 83 06 00 40 00  |.@....@.......@.|

And here's a hello world compiled with vanilla Lua 5.3:

00000000  1b 4c 75 61 53 00 19 93  0d 0a 1a 0a 04 08 04 08  |.LuaS...........|
00000010  08 78 56 00 00 00 00 00  00 00 00 00 00 00 28 77  |.xV...........(w|
00000020  40 01 0f 40 2f 74 6d 70  2f 74 65 73 74 2e 6c 75  |@..@/tmp/test.lu|
00000030  61 00 00 00 00 00 00 00  00 00 01 02 04 00 00 00  |a...............|
00000040  06 00 40 00 41 40 00 00  24 40 00 01 26 00 80 00  |..@.A@..$@..&...|
00000050  02 00 00 00 04 06 70 72  69 6e 74 04 0e 48 65 6c  |......print..Hel|
00000060  6c 6f 2c 20 77 6f 72 6c  64 21 01 00 00 00 01 00  |lo, world!......|
00000070  00 00 00 00 04 00 00 00  01 00 00 00 01 00 00 00  |................|
00000080  01 00 00 00 01 00 00 00  00 00 00 00 01 00 00 00  |................|
00000090  05 5f 45 4e 56                                    |._ENV|

So they start out the same but rapidly diverge.

I'm not sure how to actually interpret the bytecode header (something to look into), but the fact that the differences start with bytes that are either 4 or 8 seems like it might be relevant if it's a byte count (although that doesn't explain why the 'xV' part starts at different offsets). Actually trying to run or decompile this produces 'size_t size mismatch in precompiled chunk' (or 'Instruction size mismatch in precompiled chunk' with 32-bit Lua). I know Lua has a bunch of constants you can change at compile time - eg to change the native number type. It might be that they've changed some data sizes to something other than the default. I'll investigate after work today...

 

[mk-fg]

I'm not sure about that. Here's a hex dump of the beginning of _GameConst.lua: ...

So they start out the same but rapidly diverge.

I'm not sure how to actually interpret the bytecode header (something to look into), but the fact that the differences start with bytes that are either 4 or 8 seems like it might be relevant if it's a byte count (although that doesn't explain why the 'xV' part starts at different offsets).

That's what I mean by "different header".
That stuff right after lua bytecode magic are interpreter constants like size_t, instruction size, etc, and yeah, they are different here, and there is a different number of them.
But you can see same difference not only between lua implementations, but also between e.g. lua.org's 5.1 and 5.2 and 5.3 - they all have different stuff there.

Further on you can clearly see constants table and chunks of code, so follows same general layout as e.g. lua.org's bytecode, but apparently from a slightly different interpreter.
Not sure why you'd think that it's something else entirely - definitely not a PE binary or binarywriter dump, not python or perl or whatever, has .lua ext, very much lua header, lua-internal stuff like _ENV all over the place, laid out exactly like lua bytecode - what else would it be?

Actually trying to run or decompile this produces 'size_t size mismatch in precompiled chunk' (or 'Instruction size mismatch in precompiled chunk' with 32-bit Lua). I know Lua has a bunch of constants you can change at compile time - eg to change the native number type. It might be that they've changed some data sizes to something other than the default. I'll investigate after work today...

Good luck!

Guess best-case is to get list of classes, with associated methods, properties and constants from that, laid out a bit more more logically (i.e. hierarchically) than what you'd get from just dumping all strings from these files... plus lua assembly code, as a last-resort way of figuring out what these do, if not obvious from name/testing.

Not sure if it's worth the effort vs just waiting a bit for docs - modding seem to be half-baked at this point, esp. wrt GoG version (which says that there'll be updates) - totally understandable, given release priorities, but that's your call, ofc.

 

[damny]

There's apparently some dev console in there but I can't figure out how to turn it on correctly. I did manage to figure out how to make it execute code though and put the result on the workshop:

http://steamcommunity.com/sharedfiles/filedetails/?id=1335979800​

Had a look at the bytecode just now and it seems like compared to current lua 5.3, it's missing the fields for sizes of lua_Integer and lua_Number. That could be a version thing though, maybe the game uses an older one. This is where we have to figure out what version it uses

Additionally, the size of size_t is 4 instead of 8. So that part is where we have to figure out how exactly their lua was compiled so we can build the same one


In the meantime I managed to get luadec to at least *try* to decompile these files. It still stumbles over some things and there are some files it can't do anything with at all, but at least it doesn't just refuse like it's supposed to when the header is wrong. Basically, I followed the instructions for building it against lua 5.3 but made these changes to lua-5.3/src/lundump.c:

1. get luadec and build instructions from https://github.com/viruscamp/luadec
   
2. replace body of fchecksize() with only LoadByte(S), no checks, to ignore the difference
3. in checkHeader(), remove checksize(S, lua_Integer) and checksize(S, lua_Number)

I did this on a Mac and I have no idea how easy this thing is to build on Windows, but there are some VS project files in there.

 

[cpeosphoros]

There's apparently some dev console in there but I can't figure out how to turn it on correctly. I did manage to figure out how to make it execute code though and put the result on the workshop

If this console could print text to an external file, it could be relatively simple just to print dump _ENV members, recursively.

That would at least give a clue about what classes are available.

 

[damny]

If this console could print text to an external file, it could be relatively simple just to print dump _ENV members, recursively.

Yeah, mods can write to files (the normal lua way using io). That's how I started, I made a mod that just dumped _G and continued looking from there.

There actually seems to be some sort of object inspector in the game. You can call OpenExamine(some value) to bring it up. I don't recommend doing it with _G though, my laptop is too slow for that. Try selecting something and then use the console to run OpenExamine(SelectedObj).

 

[StrategicWarlord]

We have the decompiled .lua's availible on our modding discord.Everything you are looking for should be in there. https://discord.gg/PsHpkzk

 

[mk-fg]

If this console could print text to an external file, it could be relatively simple just to print dump _ENV members, recursively.
That would at least give a clue about what classes are available.

Yeah, that's the introspection option, and indeed it works, as also mentioned above.
Here's an example script (attach as "Code" to a mod) to dump stuff (e.g. all about SolarPanel) that I've used and its output:
https://gist.github.com/anonymous/987ef87babc67a591051bee349e16b5a

In the meantime I managed to get luadec to at least *try* to decompile these files. It still stumbles over some things and there are some files it can't do anything with at all, but at least it doesn't just refuse like it's supposed to when the header is wrong.

Did my poking at the bytecode with ChunkSpy53.lua, which is included in same luadec repo, which might be easier to tweak/use as it doesn't need (re-)compilation.
Sizes are configured on top there, in CONFIGURATION, and jumping around mismatched locations can be done by adjusting "idx" var.
Same as luadec though, plenty of mismatches, needs work to get anything interesting out of it.