Home Trending Latest threads New posts Developer diaries

Network Extensions - Assets bundle

Jump to latest Follow Reply
  • We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
Cities: Skylines 6597 Support & Bug Reports 472 Suggestions & feedback 3279 Modding & Resources 1954 Official Information & Announcements 154 City Journals and Let's Plays 503 Cities: Skylines Console & Windows 10 Edition 1118
Show only dev responses
K

Katalyst6

Private
6 Badges
Jul 23, 2015
12
10
  • Cities: Skylines
  • Cities: Skylines - After Dark
  • Cities: Skylines - Snowfall
  • Cities: Skylines - Mass Transit
  • Cities: Skylines - Parklife
  • Cities: Skylines Industries
Hi!
I'm the creator of the Network Extensions Project.
After created (only) 12 new roads, we have come to realize that it takes a LOT of assets (textures and meshes mostly) to create each roads (600 assets for 12 roads...).

We are currently using a custom process to load each textures (via unity) and each meshes (via ObjUnity3D). A simple process that is taking more and more time for each road that we are implementing.

We went throughout the Unity3D documentation to noticed that we could bundle all those assets together to accelerate the loading process.
We have managed to create the bundle so far (with the Unity IDE), but haven't succeeded in loading the bundle.
Could anyone guide us to implement that loading process?

Here is the code we have so far:

Take 1
using (WWW www = new WWW(BundleURL))
{
yield return www;
if (www.error != null)
throw new Exception("WWW download had an error:" + www.error);

AssetBundle bundle = www.assetBundle;
// ^^^^^^
// bundle is null!!!!!!!
}

-----------------------------------------------------------------
Take 2
using (WWW www = new WWW(BundleURL))
{
yield return www;
if (www.error != null)
throw new Exception("WWW download had an error:" + www.error);

AssetBundle bundle = AssetBundle.CreateFromMemoryImmediate(www.bytes)
// ^^^^^^
// Can't read the header?? (of a bundle made from Unity IDE??)

}
 
  • 4
Reactions:
A working feature to bundle assets would be very much appreciated. In SC4, such a feature cut game load times in half. At the moment, C:S takes about 7 minutes to load my current city, starting from the main menu. It would be nice to shave something of that off.
 
  • 3
Reactions:
I second it and Cities XXL has like All American Gas Stations that somebody has made it so somebody could take all Gas Stations so far in Steam Workshop put them in Mode same with Naming Tools,Fast Food Places,Retail Chains,Maps from a certain State and so on.
 
The programmers at Colossal are super busy at the moment but I will get back to when I have a chance to discuss this in detail with them. Thanks for your patience!
 
  • 5
  • 1
Reactions:
Hi Katalyst6,

Sorry for the delay getting back to you and thank you for your patience. martsu wasn't kidding when she said we have been busy ;)
So I am not sure what is your progress regarding asset bundles since it has been a while but even tho I have not tried that myself, it should be theoretically possible to bring assets to the game using Unity asset bundles.
Judging from the code you have pasted, it is a bit difficult to guess what could be wrong.

Could you elaborate on the process you used creating the asset bundle in Unity?
What bundle url are you using? Are you trying to load the bundle from the local drive or over http? Remember the format of a local path needs to look like "file://C:/Game/testbundle.test" i.e
And finally, where in your mod are you trying to run that code?
Depending on which method/which thread you are running this code, there is no guarantee the thrown exception will land in a place you will see in the log so you might miss the cause for the download failing.
I will try to help you further after I hear more details on how you have been doing this so far :)
Cheers
 
  • 4
Reactions:
Hi co_damsku,
First, let me thank you for taking the time to answer to my inquiry, it's really appreciated!

Let's get into business:
Before heading futher in the "bundling" direction, let me ask you a simple question: what is the best way to package/import a huge bunch of meshes (obj) and textures (png)? - around 700 assets

Right now, we have been using custom code that create Texture2D and Mesh.
Even though that process has been proven reliable, it is quite heavy on the game loading process.

What is the Normal/Optimized way you guys from CO are using to do this operation?
Could we have a simple "Crash Course" on it, to make sure we are using the right process?


Again, many thanks in advance =)
 
Hi,

No worries, here is my 2 cents on your options :)

The asset bundle way might be the easiest and most flexible way to deal with those, it will likely be faster to load than reading objs and pngs directly due to the format in which these will be stored in the bundle. However Unity asset bundles were created primarily for the web player to stream packages from the net, so I have no idea how much overhead there might be if any compression is used and things like that.
Advantages are asset bundles should be capable to package animations as well, which means you could most likely import animated meshes through them which we do not support at the time being, also Unity mesh importing is doing quite a lot of mesh optimizations so the mesh stored in an asset bundle will likely be more suitable for real time rendering.

Another option to bundle simple meshes+materials+textures would be to use our Package system to create a CRP containing all of that and just deserialize the objects when you need to set their scripts, shaders, materials and values at load time.
The namespace ColossalFramework.Packaging of the ColossalManaged assembly should allow you to do that. Unfortunately, there is no public documentation for this as this is our internal packaging system and it is primarily tested for the use cases we have in the game, so it might not behave as you would expect or have limitations if you try to use it in different scenarios.

At the end of the day tho, bundling all these assets in one file makes it easier to redistribute but does not significantly improve your loading times, what really does is the format in which they are stored at the time of loading.
The bundling process, both for asset bundles and CRP packages, will essentially compress the textures (and generate mipmaps if needed) to a DXT format which is the same than uploaded to video memory, and the mesh format does a similar thing (do processing such as triangulating, getting rids of degenerates, consolidating etc.. and store raw index, colors, uvs, vertices arrays...) so that loading is essentially one large binary read from the package per asset and no additional data crunching nor parsing is necessary at loading time.
Reading a text based model format and parsing it is much slower than reading a binary chunk which essentially only need to be assigned to the right "Unity field" in the mesh class and same goes for textures where we just load the raw image data into the Texture2D.

So yea, using a more efficient storage format for your assets is the key to improved performances so it is probably more down to your taste and patience than anything else if you wanna use asset bundles, CRPs, home made format or maybe even 3rd party packaging solutions.
Cheers !
 
  • 5
Reactions:
Ok great!
I'll try to make an executable tool out of that library that will be able to create a crp file on-demand from a repository.

Is there a place/web site I could document while I go and discover the process?

Many thanks
 
Aww. Making a standalone exec using our libraries is unlikely to work due to the fact the library is quite dependant on Unity and the way they also load our native code.
Making a Unity editor tool might be doable but as mentioned earlier, there might be unexpected behaviour or they may not work at all given our libraries are mostly tested in the context of how they are used in skylines so the way they are loaded need to be very specific since we have both native and managed code for those.

Anywho, the most likely scenario to get somewhere if you intend to use our internal stuff would be to make it as part of your mod (probably just locally). For example, load everything your usual way (using obj and unity api) and create the meshes and textures, and have an option button or a debug menu for "Pack data" in-game. When invoked, go thru your assets types, create a package, add assets to it and save to disk. That would allow you to create CRP which will be added to the package collection as long as it is in our of the game folders which automatically loads CRP files (afair you can bundle a crp file alongside a dll by copying it to the workshop staging area while sharing a mod and it most likely will be loaded for you).

Code: 
//something like that maybe
Package p = new Package("NetExtension");
p.AddAsset("Mesh1", unityMeshObject);
p.AddAsset("Texture1", unityTextureObject);
p.Save();

Once your package is loaded by the game, it's index table will be available to the game, so you can find an asset by using PackageManager.FindAssetByName() using the name you provided in AddAsset (make sure they are unique). This will return a Package.Asset with a ready stream to read and instantiate by doing Texture t = asset.Instantiate<Texture2D>() and that should leave you with a unity object ready to use.

Also serializing Unity textures which are already in video memory might not work out so good depending on their format. So you can always load most variants of png, tga, bmp, jpg and dds using our Image class (Image img = new Image(path) found in the ColossalFramework.Importers namespace of our ColossalManaged assembly). You can use that class to compress textures to DXT format and serialize them using one of our Package.AddAsset overload.

Anyhow, this is roughly how we serialize, deserialize and package/share our save games, assets created in the asset editor, theme editor, maps etc... I hope this helps a little bit.

If you really are brave enough to go on and give it a try with our own packaging system, I would be personally very interested in hearing the issues you have so we could fix them in our framework on the long run and I will try to assist as much as I can, but do bear in mind my days have way too little hours to do everything I wish to do ;)

To answer your question: I believe http://www.skylineswiki.com/Cities:_Skylines_Wiki could be a suitable place to share your experience while developing your mods, I am not aware of a very popular place designed for that purpose but I am sure creating a thread of the Paradox forums and update it with your progress would also be of interest to many people !
Best of luck and cheers for making awesome mods! :)

Cheers
 
  • 6
  • 1
Reactions:
Wow! Thank you for all the efforts you have put in those explications!

I really liked your idea of the DataPackingMod, that was the only concept we hadn't tried yet.
We will do a proof of concept (POC) of it in the upcoming days.

Thank you for your time, you guys have been a great help!
Cheers!
 
  • 2
  • 1
Reactions:
I wonder if packaging multiple assets (buildings, props) into one CRP file would decrease the loading time. That might be an interesting options for large themes.
 
  • 3
  • 1
Reactions:
Could this be used to create something similar as the .dat packer tool we had in SC4. That tool was great to reduce loading times. (I think that is what boformer is also hinting at in his post above :p)
 
I have a problem with Network Extensions Mod.
The problem is that when get into the game i don't see the new roads that were supposed to come with the mod.
I checked my content manager and the mod was on. I have no idea how to solve this problem.
I am using patch 1.4.4f2
and i have after dark and snowfall DLCs.
No other mods were running as i unsubscribed all of them.
PLEASE HELP !!!!! :(:(:(
There is an image with this, hope it helps.
 

Attachments

  • Screenshot (3).png
    Screenshot (3).png
    1,9 MB · Views: 976
have you actually enabled the mod in the mod menu? just subscribing to it isn't enough
 
You must log in or register to reply here.
Top Bottom
Back