Home Trending Latest threads New posts Developer diaries

Hacking the TBL files

Jump to latest Follow Reply
  • We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
Show only dev responses
Okay, small update. I have managed to find some time to continue work on this. Basically, I'm building a library modify the EU2 files. At the moment, all it does it read stuff, but there are "provisions" to add writing too. This includes the text files (eg provinces.csv).

Regarding editing the map: I've said before it is really hard. I've been fighting with figuring out an algorithm to compress an image to the TBL format, but again: fucking hard. :)
 
Inferis said:
Okay, small update. I have managed to find some time to continue work on this. Basically, I'm building a library modify the EU2 files. At the moment, all it does it read stuff, but there are "provisions" to add writing too. This includes the text files (eg provinces.csv).

Regarding editing the map: I've said before it is really hard. I've been fighting with figuring out an algorithm to compress an image to the TBL format, but again: fucking hard. :)
Click to expand...

In Inferis we trust.
 
Inferis said:
Okay, small update. I have managed to find some time to continue work on this. Basically, I'm building a library modify the EU2 files. At the moment, all it does it read stuff, but there are "provisions" to add writing too. This includes the text files (eg provinces.csv).

Regarding editing the map: I've said before it is really hard. I've been fighting with figuring out an algorithm to compress an image to the TBL format, but again: fucking hard. :)
Click to expand...
You know, the more information you can give us on that website of yours, perhaps the more somebody will go on the forum, read this thread, read your website and might have a good idea or two... hmmmm???? ;)
 
Could it be so, that the images on .tbl files are with a standard compression?

Too bad that there are many image file standards.
 
Okay folks, time for a question. How would you take a go at editing the map? What set of tools would you develop (supposing you have the ability) to do it?

Keep in mind that:
  • The total mapsize is 18944x7296. This results in an image that 395,4 MB big. An editor should be usable also for folks who cannot afford gigabytes of memory (which isn't much of a problem for Paradox, I guess).
  • Several files define how stuff works: the lightmap files contain the image, but there's also the id grid, the TI, the province list, boundboxes, ... All these have to be updated a the same time
 
Can you perhaps make an editor load only 1 part of the map - say 1 continent - at a time? Otherwise, it's a damn big map to load every time you want to do some minor tweak...
 
Hive said:
Can you perhaps make an editor load only 1 part of the map - say 1 continent - at a time? Otherwise, it's a damn big map to load every time you want to do some minor tweak...
Click to expand...

Was my first thought as well, but I guess it has to be saved every time? And if full file is close to 400Mb dividing by 5 will still be 80 Mb?

Maybe divide ot into 10-20 (assuming all other files needing modification are small?) and then make some sort of 'compiler' for the map?
 
Nikolai II said:
Was my first thought as well, but I guess it has to be saved every time? And if full file is close to 400Mb dividing by 5 will still be 80 Mb?

Maybe divide ot into 10-20 (assuming all other files needing modification are small?) and then make some sort of 'compiler' for the map?
Click to expand...

Good idea... extract the main map into 10-20 small chumps, which can be edited alone.... and then, when you want to use the map - you compile them together.
 
Inferis said:
Keep in mind that:
  • The total mapsize is 18944x7296. This results in an image that 395,4 MB big. An editor should be usable also for folks who cannot afford gigabytes of memory (which isn't much of a problem for Paradox, I guess).
Click to expand...

If the technical solution to that problem is working with only a smaller a part of the map at a time, I would recommend dividing the map in several overlapping parts (overlapping so You could choose a conveniant view to work form according to Your needs). E g part1 :North America, part2: souther part of NA and northern part of South America and part 3: SA. Etc.


Inferis said:
  • Several files define how stuff works: the lightmap files contain the image, but there's also the id grid, the TI, the province list, boundboxes, ... All these have to be updated a the same time
Click to expand...

Ideally You should (?) have a choice to either work with the map whithout considering the provinces (province borders) or in a "province mode" where the current "province data" is always present (editable, maybe in a floating box) when You are editing the map.


BTW, have You rellay deduct the format of the TBL files ? Fantastic !!

:) :eek: :) :eek: :) :eek: :)
 
eon said:
If the technical solution to that problem is working with only a smaller a part of the map at a time, I would recommend dividing the map in several overlapping parts (overlapping so You could choose a conveniant view to work form according to Your needs). E g part1 :North America, part2: souther part of NA and northern part of South America and part 3: SA. Etc.
Click to expand...

Or if a 'compiler' is possible, then maybe also a 'cutter' to take the map to pieces, allowing for at least two different modes of cutting (that places borders differently?)

Or cut up into 20-30-40 pieces and allow 'editor' to be able to load two adjacent pieces at once if needed?
 
A compiler would most probably be the way to go. I've been thinking about that for a while now.

Basically, the system I have in mind contains:
* a decompiler, to transform the current EU2 files into a format of our own we can use to edit (maybe just one (very) large file, but easier to maintain).
* an editor to edit the map
* a compiler to transform our own format into EU2 readable files.

This would allow you to have multiple working copies of a map (given enough diskspace, but that's not so much of an issue as memory is), compiling them as needed so they can be used to play.

The editor would be used to edit stuff like sprite placement, actual pixel ownership (the stuff stored in id.tbl), province names, ... The image (or part of it) would be exported to a TIFF or so, which can be edited in a graphical editor (eg photoshop). No need in duplicating functionality. ;)

Any thoughts?
 
probably a very stupid and impossible suggestion :)

There are a few "ID maps" floating around the forums, I don't think these, even when fitted together (I have separates for all continents) would put too much strain on any PC. now, if it was possible to select, in such a map, a certain region (by "cutting" ) that you want to edit, this region is then taken into the compiler/editor/... , the rest keeps unchanged.

that way, everyone could pick a size fitting to their current needs, and to hardware available.
 
ForzaA said:
probably a very stupid and impossible suggestion :)

There are a few "ID maps" floating around the forums, I don't think these, even when fitted together (I have separates for all continents) would put too much strain on any PC. now, if it was possible to select, in such a map, a certain region (by "cutting" ) that you want to edit, this region is then taken into the compiler/editor/... , the rest keeps unchanged.

that way, everyone could pick a size fitting to their current needs, and to hardware available.
Click to expand...

This sounds awfully complicated though, imo...

Btw, Inferis - will you also included in your editor the possibility to determine where habour icons, capital icons, manufactory icons etc are placed in a province? Would be heaps helpful...

Of course, that would require you to also use province.csv - but that's an easy excel file anyway, would that be a problem?
 
Hive said:
Btw, Inferis - will you also included in your editor the possibility to determine where habour icons, capital icons, manufactory icons etc are placed in a province? Would be heaps helpful...

Of course, that would require you to also use province.csv - but that's an easy excel file anyway, would that be a problem?
Click to expand...
No, that's the plan. That info is by far the easiest to modify.
 
if we can decompile/convert a region of a tbl file into a large lightmap picture (a simple grayscale bitmap) and another "bitmap" with a province id for each pixel (and propably something for that TI boundaries file)then that grayscale bitmap is editable in any raster image editing progam
then we need to program a province ownership (and TI boundaries) editor which should be quite straightforward hopefully
before compilation we use colorscales.csv to pack that grayscale image back to 6bit "colors" and the compression should be quite straightforward (that is, if i haven't missed something important :D)
for each block we construct the province id list first and check if all pixels have the same color and same owner(province) in that block to build that quadtree

in short this is the same thing you already posted (with a few additions maybe)
the main problems from my point of view:
  • can we modify only part of the map at a time? (i studied your documentation a few months ago but i don't remember the format of all other files and the format of incgrid.tbl is still somewhat unclear?)
    maybe it would be possible to decompile and edit a part of the map and when compiling it we alter some indexes only in other parts
  • i didn't quite understand those river id quirks in lightmap.tbl
  • everything has to be done using raster graphics
  • i missed something crucially important :D
 
Ah, somebody how knows what I'm dealing with. :)

Editing parts of the map should be feasible, but compiling should always happen for the whole map. There's too many files to modify, and it's going to be easier to do the full map at a time.

Editing the image should be done with an image editing program, indeed. We'll just have an import and export function to the EU2 map format.

Province ownership is not important in the image data when using our own temporary format. We do need to create it when compiling the EU2 map data, fcourse.

Basically, my plan is to create an editor that uses it's own file format that encapsulates all map data into one (big) file. You'd be able to import the current map from EU2 based files, and export it back to those file. But I'll be focusing on creating new maps first.

The big problem is that a lot of this stuff is interconnected, and that makes it hard to create a good object model.

And about the river id things: I don't quite get it myself. I know how it works because of the code Johan provided me, but I don't see the direct use in it, to be fair. Maybe Johan can enlighten us. ;) I guess it has primarily something to do with border drawing.
 
You must log in or register to reply here.
Top Bottom
Back