Babylon.js: Add TexturePack support (Texture Atlas + Json file)

[Nodragem]

Hello,

It seems to me that the current TextureAtlas object is lacking the ability to extract sprites and/or animations from a single image file (in jpg, png or else) and a companion json file (containing the details of where are the sprites in the image file).

The technique is used widely, and one of the most famous format/standard is from the following Texture Packer software: https://www.codeandweb.com/texturepacker

Although BabylonJS is not a 2D engine, that is still very useful for menus for instance !

Note that libGDX is also providing a texture packer, in case your are interested in coding one in javascript for BabylonJS, Phaser and other JS engines: https://github.com/libgdx/libgdx/wiki/Texture-packer

Feature request

Add import of animations and sprites from a SpriteSheet + JSON file pair

add a texture packer (but that's probably out-of-scope): Moved to #6877

[pandaGaume]

Hi All, Any information about the achievment of this task/issue. I could help but want to avoid double work.

[deltakosh]

Pinging @PolygonalSun

[briantbutton]

I have a definitional question about what folks would like the TextureShader to be. If anyone is watching, I would be interested to read their responses.

Currently, BABYLON allow textures to be imported but the application is a matter of careful measuring. Like this: https://www.babylonjs-playground.com/#ICZEXW

That is awkward and brittle. It would be better to be able to pluck sprites out by some previously affixed label or index. This label/index is presumably available from a "companion file", along with coordinates, by the "texture packer". The TextureAtlas object would make the mapping from label/index to coordinates.

Warm? Warmer?

Floating in the ether somewhere are file formats for texture packers which built the aforementioned "companion file". TextureAtlas should be able to read and use one of those formats. Presumably a popular one.

Is this in the ballpark?

[pandaGaume]

already developed a texture pack extension which support all the standard companion file format as loader. did not had the time to integrate into the framework. it's actually work as standalone bundle which add texturepack support to Babylon.
Babylon team is aware of this.

sent from Mars rover.

---

From: Brian Todd [e-mail]
Sent: Tuesday, September 24, 2019 1:40:30 AM
To: BabylonJS/Babylon.js [e-mail]
Cc: Guillaume Pelletier [e-mail]; Manual [e-mail]
Subject: Re: [BabylonJS/Babylon.js] Add TexturePack support (Texture Atlas + Json file) (#5314)

I have a definitional question about what folks would like the TextureShader to be. If anyone is watching, I would be interested to read their responses.

Currently, BABYLON allow textures to be imported but the application is a matter of careful measuring. Like this: https://www.babylonjs-playground.com/#ICZEXW

That is awkward and brittle. It would be better to be able to pluck sprites out by some previously affixed label or index. This label/index is presumably available from a "companion file", along with coordinates, by the "texture packer". The TextureAtlas object would make the mapping from label/index to coordinates.

Warm? Warmer?

Floating in the ether somewhere are file formats for texture packers which built the aforementioned "companion file". TextureAtlas should be able to read and use one of those formats. Presumably a popular one.

Is this in the ballpark?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/BabylonJS/Babylon.js/issues/5314?email_source=notifications&email_token=AFM3ZMGITELHJVA25NSFY2LQLD5Q5A5CNFSM4FZL22T2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7LVK5A#issuecomment-534205812, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFM3ZMHAYC2CVGZMX2Q57ULQLD5Q5ANCNFSM4FZL22TQ.

[deltakosh]

I shared your code to @briantbutton

[pandaGaume]

great news! hope he will be able to do something interesting with.

sent from Mars rover.

---

From: David Catuhe [e-mail]
Sent: Tuesday, September 24, 2019 2:22:19 AM
To: BabylonJS/Babylon.js [e-mail]
Cc: Guillaume Pelletier [e-mail]; Manual [e-mail]
Subject: Re: [BabylonJS/Babylon.js] Add TexturePack support (Texture Atlas + Json file) (#5314)

I gave your code to @briantbuttonhttps://github.com/briantbutton

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/BabylonJS/Babylon.js/issues/5314?email_source=notifications&email_token=AFM3ZMAIT52Y7GPPRHYBQWTQLECNXA5CNFSM4FZL22T2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7LZN4A#issuecomment-534222576, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFM3ZMGIFFTGJ3BSQ6WLRTDQLECNXANCNFSM4FZL22TQ.

[deltakosh]

To clarify, I extracted the TexturePacker to its own issue: #6877

[briantbutton]

It seems that the module we would want to enhance for this ticket is Sprites (https://doc.babylonjs.com/babylon101/sprites). Yes? No?

I cannot think of any reason to leave Sprites as they are, with partial support for the external TexturePacker utility and building a new, different utility in BABYLON. Please gently let me know if I fail to understand something.

[deltakosh]

Agree!

[Pryme8]

So after doing so looking around under the hood and trying to be a cool kid and deploy my own sprite map with actors I was able to establish what feature is lacking from the current system and some flaws with how we are rending them to the screen.
https://playground.babylonjs.com/#X2FVF5#3 <- Arrows Move +- Zoom
The top row is the standard Sprite system. I had to hack it a little bit to account for the scaling changes. The bottom rows are a quickly deployed PerFragment Sprite Shader that is rendering on quads just to demonstrate how the base system should look.

I am going to purpose some changes and additions but I need some support here on how to keep the backward compatibility. I want to add all of the features of the TexturePackers standard JSON from into the Buffer that we are passing the Sprite system

//Mat4 Frame Structure
//[0] : Frame xywh
//[1] : spriteSourceSize xywh
//[2] : sourceSize xy rotated trimmed
//still have 4 slots to do things with

Is the structure I use when I do PerFragment Sprite stuff and our current shader seems to have quite a bit of that ready to go in the attribute buffer. So I am coming to the assumption is that the error is in SpriteManager. Im particularly dubious of lines
https://github.com/BabylonJS/Babylon.js/blob/ef59cbec46fd1800de0c3578b053d8361b635fca/src/Sprites/spriteManager.ts#L280
to 292.

The plan is to make these things render in the correct position, a 1x1 sprite should not show a seam between them when placed in the correct spots. And then make sure that the clipping system and position data passed from the JSON file actually supports more than just the frame. Oh and I gotta add the Parser for JSON Array Type and have the Sprite Manager auto-detect which type of JSON it is.

Ummmm anything else?

In addition, I want to deploy a SpriteMaterial which will be a Shader for rendering out more specific sprites who might need lighting effects normals maps (non billboard) etc, which will all be from #include stuff...

[deltakosh]

First I think that we need to modify the structure ONLY if we have the data so we need (Look at what I did for Particles for instances. I use #ifdef in the shader to make sure no extra data is processed if not required)

You plan seem fine to me. The core priorities are:

• No impact of perf if the feature is not used
• No backward compat issue

[Pryme8]

I also need some suggestions on how to handle non-concurrent frame animations. Otherwise, support for animations on "trimmed and rotated" Atlases will be hard to pull off do to the auto arrangement that happens in the TexturePacker program.

I would be leaning toward a support texture buffer that holds the animation cells and timings that is auto-generated when some new method like "sprite.addAnimationFrame" would take care off. That way we could dynamically change the offset of the animations have different timings for frames and maybe even support rotation origin.

Also, I was mistaken in my last post, the epsilon is not a problem, I just had to set it to 0. I am thinking we should default it to 0 as anything else offsets the mesh from a real position.

[deltakosh]

I would be leaning toward a support texture buffer that holds the animation cells and timings that is auto-generated when some new method like "sprite.addAnimationFrame" would take care off

I like it!

[Pryme8]

I was starting to see limitations to the workability of the current sprite system to create "levels"
The reason for why is clear with these two PGs:
https://playground.babylonjs.com/#X2FVF5#11
^ is the classic system rendering out 200by200 sprites. Pretty laggy, because at this point there is a ton of vertex data. Not ideal for a Sprite Map level. Great for characters and actors on a sprite stage because they all have their own movement.
https://playground.babylonjs.com/#3X5XME#8
^Is a texture lookup system, doing 2000 by 2000 Sprites. Great for level design, Not ideal for actors as it's restricted to a grid. Can and will implement lighting effects because we have fragment data.

_I need some input on how to make the structure for the animation texture._
Ideally, we could pass the cellID so we can get the frame info from the frame map, and then have it increment through the next frames at certain time intervals. But I am not really sure how to handle that as of yet... I kinda had a hacky version in the past but wanna work up something more legit.

Also, did we ever get the 9 Patch Support?

[deltakosh]

The second PG does not work :(

We do support 9patch for images in the GUI not for sprites.

I need some input on how to make the structure for the animation texture.
Can you elaborate more?

[Pryme8]

My bad! I actually figured it out. I created a lookup texture method that makes it so you can assign CellID and custom timing for each animation and frame. With an additional value still blank in case something comes up.

Now I am modifying the frame lookup texture to account for rotation and trimming and will be fixing this up a little bit.

If I can get my command line to stop being a punk ill push the changes to the standard SpriteManager to account for the Frames.Array parsing it is lacking.

I also gotta figure out why it's not working when the scene loads from an outside link... but works when you copy and paste the code into a fresh pg.

[Pryme8]

https://www.babylonjs-playground.com/#958CJY#1

Here you go, had to stick everything in the createScene function

[Pryme8]

How does the trim parameter work with the TexturePacker?
Here is the format:

"frame": {"x":1,"y":103,"w":24,"h":32},
"rotated": true,
"trimmed": true,
"spriteSourceSize": {"x":4,"y":0,"w":24,"h":32},
"sourceSize": {"w":32,"h":32}

Ive got the rotation and frame figured out, I just dont know how to incorporate trim now.

mat4 frameData = getFrameData(frameID); 
vec2 frameBase = frameData[2].xy;   
vec2 frameSize = (frameBase)/spriteMapSize;
 vec2 offset = frameData[0].xy*sheetUnits;  
//rotated
if(frameData[2].z == 1.){
    tileUV.xy = tileUV.yx;
}           

color = texture(spriteSheet, tileUV*frameSize+offset);

[deltakosh]

Sorry but I'm a bit under the water for now and I cannot help too much :(

[deltakosh]

(not a big deal if for v1 we do not support trim)

[Pryme8]

I was about to ask that. hahah!

[deltakosh]

Pinging @Nodragem: Do you feel like the PR #7198 works the way you expected it? Do you feel like having a new class named SpriteMap is ok for your needs?

It is not generating a sprite manager and does not generate sprites. There is also no picking because of that

The good point is that SpriteMap does not breack back compat and it is very fast. So a good companion for dynamic sprites