Maximum filesize on workshop?

[Walshicus]

Hi all, is anyone aware of a maximum filesize for a mod uploaded on the Steam Workshop? We've had issues lately when exceeding 2gb - on windows the archive appears to get corrupted with a weird descriptor and on Linux it doesn't look like it downloads at all. 2 gigs is a pretty low ceiling to work within... :S

 

[Grinsel]

we think its same issue like 0.3.0 500MB border for mods. so we reached not 2.x GB...the next border...

we bodly go where no mod has gone before...we need SPACE!!!!

greetings

 

[FullMetalFox]

2 gigs is a pretty low ceiling to work within... :S

For most mods that is sufficient, and considering that Stellaris is running on a 32bit engine, one could even argue that it encourages much need file optimization, something most modders disregard. (Slight hint: New Horizons needs some improvement in its file optimization, especially with partially blury 2048 textures wasting not only disk space, but precious memory space which is quiet limited on a 32bit engine)

 

[Shaggo]

even with significant optimization 2 gb is not nearly enough for our vision and it works just fine on the local version.

 

[Grinsel]

remember: every version is local, so if u have the files it works fine. only the upload-limiter of the mod-app from pdx is our problem.
im sure that the pdx team that i love since 2006, will be careful and step by set open the borders of the game for the modding community.
that's smart and very thoughtful, but now it's time again to open the border of stellaris modding. we all love the game. we love the crew we love pdx.

open it asap, our patches are ready.

greetings

 

[Walshicus]

For most mods that is sufficient, and considering that Stellaris is running on a 32bit engine, one could even argue that it encourages much need file optimization, something most modders disregard. (Slight hint: New Horizons needs some improvement in its file optimization, especially with partially blury 2048 textures wasting not only disk space, but precious memory space which is quiet limited on a 32bit engine)

That's a very good point, we were being very inefficient with certain assets. Thanks for the feedback - and for now - a solution!

 

[mightygaz]

Hi all, is anyone aware of a maximum filesize for a mod uploaded on the Steam Workshop? We've had issues lately when exceeding 2gb - on windows the archive appears to get corrupted with a weird descriptor and on Linux it doesn't look like it downloads at all. 2 gigs is a pretty low ceiling to work within... :S

Can't you just split it if needed? part 1, part 2, and so on. enable both, job done.

 

[Grinsel]

thats a way to live with that size-limiter, but we wont a limit...its easyer for everyone thats the mod is able by `one-click` on steam.

i am hate mods with such needing submods...and its very dificult for a bunch of files that the models need are in every single submod.
its possible, but hard to code.

but...thx linux i had can use a small code to resize some of our files that we can work on, till the patch from pdx open the size limit.

greetings

 

[FullMetalFox]

even with significant optimization 2 gb is not nearly enough for our vision and it works just fine on the local version.

Last time i checked, you were using around 1.7 gb for models and textures, the most significant cut being the textures themself. Now you are using 2048x2048 textures for models which most of the time dont even fill half the screens resolution, and im guessing your using the same size for normals, spectrals and emissive maps too. That is rather wasteful and over the top.

And if something works right now or will work in the future are two different things.
Youre working on an Engine that is subjected to the 32bit memory limit, aka 4gb. On average it already uses up around 0,9gb just to run a game, with larger maps, large battles, and the more unique stuff is on screen, that number will increase. Ive had peaks of around 2.5gb in major multi species battles in vanilla, especially in close up views.

Now in a future version of new horizons with even more models, it could be very likely that you will cause out of memory issues just by quantity.
So as harsh as it may sound, you may have to adapt your vision to the limitations of the engine youve choosen to work with.

Just to make it a short list, optimization will do the following:

Pros:
1: Prevent future memory space issues.
2: Allow you to work with the current workshop limits.
3: Allow players with les powerful machines to enjoy your mod.
4: Reduce load times.
5: Reduce load stuttering.

Cons:
1: Slightly reduce quality on very close zoom levels.
2: Force some extra work on your Artists.

That's a very good point, we were being very inefficient with certain assets. Thanks for the feedback - and for now - a solution!

No problem, im curious to see were new horizons journey will lead it.

 

[Grinsel]

u said it, 4 GB useable but 2 GB the size limiter, its much more space to work on.

normal and specular rezised from 2k to 512px.
nobody can say thats is wasting space!
both scenes with same 2048px diffuse



2048px normal and specular

512px normal and specular

its not WASTED!

but, of course, we are wasting space in the texturing files, unfavorable distribution is the point.

but we are smart, and find any good solution, with the help of this community i guess.

thx

 

[Shaggo]

indeed the quality difference is pretty huge on some ships. anyways we have managed to cut down our mod size in half, but it wont take us long to get to the cap again...

 

[FullMetalFox]

Ive also said that nearly 1 of those 4gb is used by vanilla stellaris on running an average game. You can go on and do some googling on how most engines do work with their memory, how loads peak on disk to gpu streaming and such. In overall: work with below the max, especially if you cant control how loads peak as you cant prevent a player from zooming in on a battle with 100+ different ships suddenly wanting to load their highes MIPs. Otherwise you can only hope that the Engine does feature out of memory preventions, which would still lead to bad performance, but atleast spare you the "out of memory, i quit" notification.

2048 to 512... Grinsel, you overdid it, thats a 1 to 16 ratio (derped, to many numbers in my head).

1: Diffuse from 2048x2048 to 1024x1024 = 2731kb to 683kb
Since its a diffuse, youll have to do some cleanup and crisping.

2: Normal/Emmisive from 2048x2048 to 1024x1024 = 5462kb to 1366kb
You should rerender normals to this resolution. Rerendering it is always more accurate than image software rescaling, expecially when your using bicubic interpolations.

3: Specular from 2048x2048 to 1024x1024 = 5462kb to 1366kb
On this you could potentially even go down to 512x512, cutting it down to 342kb, but that depends you your speculars detail grade. Rule of thumb: the more grundgy/dirty/detailed a surface is, the closer a speculars resolution should be to the diffuse's, the smoother and cleaner a surface is, the more you can scale it down.

So just with 2048x2048 to 1024x1024 you can cut your load down from 13.655mb down to 3.415mb, thats already a 75% size reduction.
And yes ill not even go on about inefficient UV mapping, because that is a very common issue and the workload requiered to improve those would propably piss of your artists, alot.

edit2: Also, keep in mind most of the time you dont view ships that close

 

[Grinsel]

from 512 to 2048 its a 1 to 16 ratio

512 x 4 = 1024 x 4 = 2048

512 x 16 = 2048

u forget its squares. the multiplyer is x4 not x2


check it by this grafic:

u loss with every stage from 2048 to 1024 to 512 to 256 to 256...everytime 3 of 4 pixels.
its a massive loss of detail.
each detail loss 75% of his informations.
sure u can new export from origin file...but the reduce is ever a information loss in ratio 1 to 4

 

[FullMetalFox]

Yeah ive edited that number shortly after you did post it, propably had some other number in mind from my star class excels. Or just not enough coffee.
All other numbers in that post were correct though.

 

[Revan67]

Hi all, is anyone aware of a maximum filesize for a mod uploaded on the Steam Workshop? We've had issues lately when exceeding 2gb - on windows the archive appears to get corrupted with a weird descriptor and on Linux it doesn't look like it downloads at all. 2 gigs is a pretty low ceiling to work within... :S

Is this a limitation set by Steam or is it Paradox? If we can be of any assistance getting this changed we will gladly lend our help. I seem to recall Bethesda had a similar situation with Skyrim in the early days of the workshop and they themselves increased the limit.

 

[Grinsel]

its pdx mod uploader limit.

the 32bit is the point...read earlier msgs

 

[LocutusvonBorg]

i love new horizons

i am asking myself why pdx didnt done it in 64bit from the start ? i mean they know their modding com and mods can sometime make up a new game ... so why did they limit it that hard ?

to the reduction of the textures: ofc 2048 looks way better, but how often you zoom in that close when regulary playing ? i only did it when i saw new ships, the rest of the time i am pretty much zoomed out. so i think 512 will work pretty good and the benefit is worth the lower resolution.

resistance is futile