Monogame: Why stick with Content Pipeline?

[RUSshy]

This is super annoying for lot of assets, sounds, etc.. and specially for fonts

Is it possible to let us load assets from stream and stay away from that content pipeline annoying thing?

Also for fonts, why not being able to simply use TTF fonts instead of that xml thing?

Something like:

var fontGen = new FontGenerator("path/of/ttf/font");
fontGen.size...
fontGen.spacing..
fontGen.outline..

var spriteFont = fontGen.Build();

Far easier and much more logical that the xml mess

I hope i don't sound too rude but its been 1 week since i try to find a good solution for spritefont without that content pipleline, i want manage my assets as i want not forced to use a specific app

[kameko]

Heavily agreed. I especially need this because I'm making a Game Maker clone and I can't just tell users to use some third-party tool just to load their assets. In fact I would have adopted MonoGame for my project sooner if it weren't for the fear of being forced to use the content pipeline. I'm really grateful we can at least load raw image files without the pipeline, but it'd be nice if we could do fonts and audio, too.

[harry-cpp]

Why it's used: http://www.infinitespace-studios.co.uk/general/monogame-content-pipeline-why/

[tomspilman]

but it'd be nice if we could do fonts and audio, too.

You can.

For fonts just call the constructor and pass the texture and all the kerning info, but it is up to you to build that information how ever you want.

For audio you have SoundEffect.FromStream().

[tomspilman]

I can't just tell users to use some third-party tool just to load their assets

They already have to use "third-party" tools to create the assets in the first place. Still it isn't like modders are not used to using tools to mod games. Look at GTA modding or Doom modding or Half-life modding... there are plenty of tools people have to use. Stardew Valley which has sold over 1 million copies on Steam used XNA and people mod the XNBs all the time.

If even then you want to do better than the GTA team or Valve did... you can. We provide all the source to the content pipeline on purpose... so you can customize it.

You can easily write a method to scan your mod folders and convert PNGs to compressed textures, WAVs to compressed audio, etc on startup of your game or even dynamically at runtime. This way you get the best of both worlds... optimized formats and mod friendliness.

[kameko]

Ah, I assumed there must have been a direct streaming thing for audio but I didn't see it anywhere. Same for fonts and images, I had to ask how to stream images on the MonoGame forums.
Maybe you could document how to load content directly instead of through the pipeline? If it's not documented, than I'd think more people like @RUSshy and I are surely to ask the same questions.
I understand the benefits of the pipeline, but my engine is a 2D engine with tiles and chunks that get unloaded if they go offscreen, so it's really lightweight. I don't get a lot of benefits of the pipeline, so it's mostly a nuisance to me for that reason. If I was doing a 3D engine that didn't chunk it's maps, I'd probably have more reason to use it.
And I know about modders needing tools, but I still don't want people to have to use tools. I'd like it to be a drag&drop solution. I was considering doing that compression-at-runtime thing, though. I didn't look into that yet. Is there any documentation on doing that? I didn't see it on the MonoGame website last I checked, but I'm pretty bad at navigating websites, haha.

[angrywasp]

Just to be very clear, MonoGame is in no way dependent on the pipeline and indeed is not dependent even on .xnb files. When i first started using monogame, the content pipeline still needed you to install xna. so i went out and wrote my own binary file formats for all of my game resources. Textures, fonts, meshes, etc, along with new resource types that aren't included in monogame. I then reimplemented the content loader to load my files and bypass the content pipeline completely.

In fact i mainly just use monogame for window creation, device handling and alike and do not use any of the "game" features of monogame.

So the bottom line really is that there is nothing locking you into using the pipeline. There is nothing dictating that you have to prebuild your assets either. But to ditch the pipeline means you have to roll your own solution and then maintain that as monogame evolves. It is actually quite simple, just requires a bit of forward planning and time to implement.

[RUSshy]

Another reason why ContentPipeline is so bad, look this framework: https://github.com/tlgkccampbell/ultraviolet

It lets you preprocess or load your raw assets at runtime, it doesn't force you something

this is reason why i switched to ultraviolet, and it lets me use GLSL shaders, i now know why Microsoft abandoned XNA

[harry-cpp]

It lets you preprocess or load your raw assets at runtime, it doesn't force you something

MonoGame also doesn't force you to preprocess your assets (except shaders)...

[Jjagg]

Another reason why ContentPipeline is so bad, look this framework

"This framework doesn't use a content pipeline" isn't really "a reason why "ContentPipeline is so bad" honestly.

It lets you preprocess or load your raw assets at runtime, it doesn't force you something

Just like MonoGame, like @cra0zy just mentioned :)
And that already came up in this thread before. Check out Texture2D.FromStream and SoundEffect.FromStream.

i now know why Microsoft abandoned XNA

I'm pretty sure it wasn't the content pipeline and it wasn't the absence of GLSL shader support :p

[RUSshy]

Exactly, Shaders / OBJ models / Fonts still rely on content pipleline, wich sucks

[tomspilman]

Exactly, Shaders / OBJ models / Fonts still rely on content pipleline, wich sucks

@RUSshy - Then maybe MonoGame isn't for you.

We not only serve users targeting PCs only, but also indies/professionals that want to ship on consoles. Using content optimized for the platform you're targeting is critical to having good load times and performance. It helps on all platforms, but benefits consoles with slow hard drives, average CPUs, and limited memory the most. This is also why you see Unreal/Unity/CryEngine do offline compiling of content.

It likely isn't any concern to you now experimenting and shipping games for PCs only, but if your game gets to be of average complexity and you look to ship on a console you'll realize why having a good content pipeline is important.

[harry-cpp]

Don't forget about shipping games on phones, most phones have very limited resources.

[RUSshy]

You missed my point

It makes sense maybe if you ship your game to console or mobile, but only if you will develop and run them on that target, wich never happen

During prototyping, testing, editing, modding phase, it is just completely stupid and pointless, iteration becomes slow and painful

It break your workflow it force you to use an buggy UI to manage your assets, and if you have ship.png and ship.obj file, content pipeline is too stupid to know that they are different files

it duplicate all your assets, raw + xnb, wtf, what is an xnb, and how you differentiate stuff, during dev phase it is stupid

Unity / CryEngine / Unreal don't have that issue cause their entire engine is around their UI, so asset management is totally transparent, unlik monogame

[harry-cpp]

If you are only targeting Desktop you could pretty easily include MonoGame.Framework.Content.Pipeline.dll and process and load raw assets during runtime, tho it would be way slower than if you were to preprocess them.

[craftworkgames]

During prototyping, testing, editing, modding phase, it is just completely stupid and pointless, iteration becomes slow and painful

I actually agree with this. Being able to iterate fast during development is a really good thing to strive for.

The Content Pipeline has it's purpose and it does it's job there's no doubt about that. However, a lot of the time when you want just want to add a texture, sound, data, map or whatever to your game it's quite a slow irritating process.

In an ideal world adding a texture to your game should be as simple as dropping it into your Content folder and calling Content.Load<Texture2D>. That's it. 2 steps.

I think what most people do is this:

1. Open the Content Pipeline Tool
2. Right-click the Content tree node
3. Click Add -> Existing Item...
4. Navigate around the file system (this is rather annoying sometimes)
5. Double click the file
6. Deal with the dialog that comes up (yes, of course I want to copy the file to the directory)
7. Finally try load it with Content.Load (oh shit, I forgot to save Content.mgcb).
8. Okay, now I've saved and it finally works.

Okay, okay. There actually is a faster way to do this if you've already got the Content Pipeline Tool open on your other monitor. You can just drag and drop your content into the tool. But that still only saves steps 2, 3 and maybe 4 (if you've already got windows explorer open at the right place).

My point is, there's definitely room for improvement. The tool could monitor a folder for new files and the UX could be streamlined quite a bit.

I get it. There's 722 other issues to deal with. Improving the Content Pipeline is probably not on the top of the list for the core team. I just wanted to throw in my 2 cents here. There's definitely a noticeable impact in the MonoGame community. Some people really, really despise using the Content Pipeline and I think it's because the user experience isn't great.

That said, we're an open source community. I'm sure someone could take this on. Maybe the right way to deal with it is to raise an issue to gather ideas and put a call out to get help implementing them. To be honest, it's something that's been on my mind for a while. I would like to get to it myself, but my to-do list is already a mile long.

[harry-cpp]

Open the Content Pipeline Tool
Right-click the Content tree node
Click Add -> Existing Item...
Navigate around the file system (this is rather annoying sometimes)
Double click the file
Deal with the dialog that comes up (yes, of course I want to copy the file to the directory)
Finally try load it with Content.Load (oh shit, I forgot to save Content.mgcb).
Okay, now I've saved and it finally works.

How would you shorten this process?

[craftworkgames]

How would you shorten this process?

Ideally it would be this:

1. Put your content file in the Content folder.
2. Call Content.Load

The very existence of a file in the Content folder should be enough for the Content Pipeline to compile it. Sure, sometimes you might want to configure the content differently but in my experience this is fairly rare and the Content Pipeline should be designed with the common case in mind. In terms of iterating quickly the defaults are usually fine.

This is not a new idea. Take a look at the way Visual Studio Code behaves and you'll find this is exactly how it works. Visual Studio is also moving in this direction with .NET Core project files. Removing the Right Click -> Add -> Existing Item... -> Select File -> Click Add flow makes a huge difference to the workflow.

[harry-cpp]

Open the Content Pipeline Tool
Right-click the Content tree node
Click Add -> Existing Item...
Navigate around the file system (this is rather annoying sometimes)
Double click the file
Deal with the dialog that comes up (yes, of course I want to copy the file to the directory)
Finally try load it with Content.Load (oh shit, I forgot to save Content.mgcb).
Okay, now I've saved and it finally works.

The only thing you shortened is step 1.

[Spool5520]

No the entire process is shortened. All of those steps are removed and the interaction with the content pipeline is within your IDE in code not the program itself. Which is really nice. The only other thing you have to do is drag and drop your asset into the folder. Done. I might look into adding something like this.. hmm

[Spool5520]

You would have to add optional parameters to the Content.Load function. For the options that you have on the asset in the pipeline tool. Such as mipmap generation, compression ect. This would actually be incredibly useful if it was able to do this at runtime. Especially if you plan on adding modding support. Which with any modern pc game is pretty much standard. Dam I really need to look into this

[harry-cpp]

The only other thing you have to do is drag and drop your asset into the folder.

You can drag and drop assets into the pipeline tool...

You would have to add optional parameters to the Content.Load function. For the options that you have on the asset in the pipeline tool. Such as mipmap generation, compression ect.

We couldn't. the content pipeline params are used during pre processing of the asset, they are not something you can just toggle on/off on the fly.

[craftworkgames]

I think both you guys have completely missed the point.

I'm not suggesting we need to process assets at runtime.
Drag and dropping really has nothing to do with it either.

The point is that you shouldn't need explicitly to add assets to the Content Pipeline in the first place. The very fact that the asset exists in the Content folder should be enough for it to be pre-processed and loadable.

Think of it this way. Imagine you've got your artist hat on working on a new texture in Paint.NET (like a pro). When you're ready to save the file you click File -> Save As and save the file directly to the Content folder.

At this point you shouldn't have to do anything else. It should be ready to load into the game with Content.Load. You don't even need to open the Pipeline tool.

That's not to say the Pipeline tool has no purpose. I understand there'll be times when you want to configure the content more carefully. Of course you can still do that when you have too. This is about speeding up iteration time by removing unnecessary steps.

[harry-cpp]

I do see your point, but the properties that you can set for the content are very important. If you can think of an idea that would allow both what you suggested and setting content params I wouldn't mind implementing it.

Also, on the side note, I actually very much dislike the Pipeline Tool, tho it is way better than the alternative of having a custom csproj project type. For my development I created a "special" version: #4816

[HopefulToad]

If you can think of an idea that would allow both what you suggested and setting content params I wouldn't mind implementing it.

@cra0zy What if, upon building a MonoGame project (and thus automatically building the content), any assets found in the Content folder are automatically added to the MGCB, using default parameters, and then built? If the user then wants to modify the parameters, they can open up the Pipeline tool and change them, since the assets were automatically added when the project was built. And maybe if the user doesn't want assets to be automatically added, they could set a flag in the MGCB file so that it won't attempt to automatically add assets.

This would allow for both the normal way of adding content by opening the Pipeline tool, and the drag and drop way of just putting it in the content folder.