Minecraftforge: [1.13 suggestion] Tag system for Oredict

[Boundarybreaker]

Minecraft Snapshot 12w49a adds a new system called Tags. In the update log's own words:

This seems really similar to the ore dictionary, and it seems like it'd be pretty easy to use for JSON recipes, along with making checking for ore dictionary matches a _lot_ easier. For example, one of my mods has to have a whole function to check if a slot can accept a certain oredict entry, but that could be shortened to oredict:<entry> or Tag.oredict.<entry>.
The main issue is that it might cause a good amount of refactoring needed to add oredict entries to the tag system, and it might not be as easy to access from the code side. I want to try to work on a PR for this, but I'd like some feedback on if people think it's a remotely good idea in the first place.

[Shadows-of-Fire]

The issue with tags instead of oredict is the need for a domain. The tag system works fine for minecraft and data packs, but not so much for cross-mod interactions.

[williewillus]

the domain can just be a commonly decided one, like ore:ingotIron or forge_oredict:dustDiamond

To my knowledge, no currently widely used mod has either of those ID's, so we can just reserve that domain.

That being said, this system is perfect for solving the "enumerating every variant of everything" problem that the flattening brings.

[Boundarybreaker]

I need to actually look at MCP/Forge for 1.13 and see if it lets you add a full data pack via the mod, since data packs are normally per-world. I know Forge does JSON recipes and stuff, but I dunno if it does the rest of the data pack-exclusive things. It should definitely handle data packs because they're a way simpler way to do some of the big stuff mods want to do, especially after the flattening. Mods could use tags for lots of stuff, like for slot validators (only let things with a certain tag into a slot, etc.). Since the tags have a format of namespace:name, Forge should probably require tags to be called as namespace:tag.name or something similar instead.

[williewillus]

well there is no forge/mcp for 1.13 yet since 1.13 isn't out yet.

[williewillus]

additionally, there'll probably be a mechanism to expose mod jars as datapacks. it's how resource packs work right now (each mod is a resource pack providing its jar assets)

[Boundarybreaker]

Oh, right. I thought I'd seen something about there being dev builds of Forge/MCP available to test out effects of the Flattening, but I guess not. They definitely haven't had enough time to integrate tags into them anyhow, so this is definitely all hypothetical.

[liach]

it's just stupid dicts. we may move oredicts to use snake case, i guess?

[Prospector]

Yeah, tags use the same case style as registry names.

Also, my suggestion would be to use the default namespace for things (minecraft:copper_ingot, or just copper_ingot) so that if Minecraft ever updated to include copper ingots, every mod would already support it.

Unique namespaces can be reserved for things like thermalexpansion:machine_frame, where other mods may have machine frames, but this one is specific to thermal expansion, meant to include the machine frame tiers.

And to add onto what Willus said about mods as data packs, that would make a lot of sense, because right now mods load things like recipes from the assets directory, which in 1.13 should be changed to a data dir

[Boundarybreaker]

That'd also work really well with @mezz's Unification PR, where all the things that work exactly the same get unified into a canonical Look (the minecraft:copper_ingot) approach, while other mods can have their own separate items that do individual stuff separate from other entries (so you could have a unified minecraft:copper_ingot but a non-unified oredict:blue_dye, so blue dye doesn't get turned into lapis which would be cheaty).

[williewillus]

I would rather we not pollute vanilla's domain though. If vanilla does come out with copper in the future, we can make changes then. IMO, it's safer to reserve our own domain like I mentioned above

[Gaelan]

<bikeshed> I'd propose using shared:ingot_copper. "Oredict" wasn't very accurate in the first place, and now that it doesn't even refer to a separate system from the other tags I feel like it makes even less sense. </bikeshed>

[chylex]

shared: sounds good, I was initially thinking global: but it's gonna come down to whichever we like best.

I also thought of using a wildcard or characters that cannot be used in domains, such as *:ingot_copper, which would be understood as "ingot_copper from all installed mods". Maybe that's way beyond the scope of this (and maybe looks a bit ugly), but I figured I'd throw the idea in.

[Gaelan]

*: gets weird with things like tofu and other items that aren't actually the same item, but count are the same in recipes.

[KnightMiner]

You are mixing up tas with an equivalency system, from what I've seen of them they are for recipe inputs. Vanilla wool types for example are not equivalent but currently has the tag, recipe outputs are better off using the system fron mezz's PR.

[williewillus]

A couple more random points I thought of:

• We should unpatch getSubItems and getSubBlocks back to their vanilla state, right now Forge hijacks those to "explode" an item into all of its subvariants, but tags fully supersede that. Where vanilla tags are deficient, forge should help by specifying more (enchanted books, potions, etc.)
• If we decide the OD should remain alongside tags, we should be able to register an OD name -> tag name, in lieu of using WILDCARD_VALUE, which itself should be removed.

[Alpvax]

WILDCARD_VALUE should disappear anyway, as item metadata is being removed (damage being moved to NBT)

[Boundarybreaker]

Oh, forgot to note:
Snapshot 17w50 added new functionality to Tags, both with them being able to include other tags and them appending on override by default.

There should definitely never be a scenario in which adding in an oredict entry replaces everything else for that entry, but that seems to only be when overriding a tag that's from a different namespace.
The "tags-can-include-other-tags" thing is really interesting, but I can't think of any way it'd be directly helpful to oredict at the moment.

[Alpvax]

The "tags-can-include-other-tags" thing is really interesting, but I can't think of any way it'd be directly helpful to oredict at the moment.

I can think of an example: gears of different tiers, and you could have a generic "gear" tag which could be used as any gear in a recipe (and then the output could be NBT modified based on the input)

[Boundarybreaker]

Ooh, that actually is a really good way to use tags! I really hope the MCP for 1.13 has a good way to use tags from codeside so all this can actually be implemented.

[Alpvax]

It does say they can be used in recipes, so I'm guessing there will be a vanilla method to load ingredients from tags. That combined with disappearing damage values and moving to NBT should mean more comprehensive vanilla support for advanced recipes (I'm assuming dyes are still a single Item, with differing NBT, but I could be wrong)

[Boundarybreaker]

Nope! Minecraft just had a Flattening; any differently-functioning items that were part of the same metadata before just got flattened out. Every dye is its own item, every plank is its own item, every slab, stair, etc. are all their own separate items now. Mojang's taking the flattening and new IDs very seriously. Here's a list of everything that's been flattened out.

[Alpvax]

So am I correct in thinking that NBT will now be used only for colourable items (leather armour, banners, shields etc.), damage values, and things like player heads (at least in regards to Items, and therefore crafting)?

Is there a list of current tags somewhere? Here

[liach]

Not really. The 16 colors are flattened, but armors and tool damage and leather armor color should still be the same.

[Boundarybreaker]

NBT is also used for any dependent blockstates. For example, you can't _craft_ a lit furnace, so that's an NBT state of the regular furnace. Lit/unlit versions of redstone are NBT, open/closed doors/trapdoors are NBT, etc. There are some kinda weird exceptions to this, like each potted plant becoming its own item and each state of an anvil becoming its own item. The "Block States" section of the list shows what blockstates are getting moved to NBT and what are getting flattened.

[Alpvax]

Yes, but block states aren't used in vanilla crafting (as crafting is always done with Items), so chances are Mojang haven't added/ improved NBT ingredient loading.

[williewillus]

Here's how they're flattening, the strategy is really straightforward:

Items:

• Does your item have tool damage? NBT
• Does your item have infinitely many variants (maps) or some sort of combinatorial explosion (banners)? NBT
• Else flatten

Blocks: for a given blockstate property,

• Can it be expressed using the same itemblock (furnace facing depends on how you place it, lit state depends on TE)? Keep it as a blockstate property
• Otherwise, if you would've had to flatten the itemblock (wools, clays), flatten the blocks.

Modders really should be preferring flattening as NBT hacks introduce lots more complexity and are not nearly as well supported by all the systems.

@Boundarybreaker a lit furnace actually has no item form since it's a blockstate property of furnace and there's only one furnace itemblock.

[liach]

Well, the very first thing is that block states are not stored in NBT. They are defined in code as some innate properties of blocks.

[HenryLoenwind]

Thx Boundarybreaker for the flattening link. Very helpful.

Does someone has some info on how blockstates are stored, now the meta has gone? Do blockids get assigned to blockstates or is there some new storage? From that list it seems there still is a 4-bit limit on the number of possible blockstates?

[Boundarybreaker]

I think the apparent 4-bit limit is from nothing using more than 4 bits previously. We'll have to actually check ourselves to really know.

[HenryLoenwind]

Some of the blocks have been split into 2 when they use more blockstates than before (heads, torches, stems). I see no good reason for this, so I'm guessing. Without someone who does snapshot-code-diving it's all guesswork...

[Prospector]

In 1.13 BlockState IDs are assigned automatically per chunk. 256x16x16 = 65536 which is the ID limit. This gives you theoretically infinite IDs, as you can have a unique BlockState in every spot in every chunk.

[LexManos]

Incorrect, they use a pallet system for simple chunks, but once things expand above 16(? Don't have the code in front of me) different types of blocks they fall back to the global registry without local palletts.
Basically, storage wise it's still the same, it's just loaded and treated as a single id internally and not two (id/meta)

[HenryLoenwind]

Thx Lex. So every block still has a 16 ID range for its blockstates and there's some mapping for them? Sounds weird, but as long as there are still blocks with blockstates, it won't effect mods at all. Aside from now every second mod having a couple of static lists for mapping dyes, wools, clays, ... to colors.

[williewillus]

States are now stored in chunks using the palette lex mentioned, but if storing them in a one-off place (entity/te) it just saves the resource location and property value map into nbt. The method they use is actually in 1.12 (see NBTUtil.readBlockState)

Mods won't be "Not affected at all" because since items need to be flattened, blocks have to be as well, because itemblocks.

[williewillus]

Also, you can have more than 16 saved variants. If you look at the 1.13 chunk format you'll see that things like fence and redstone connections are now completely saved

[Prospector]

Note block also has a fuckton

[HenryLoenwind]

williewillus, I was talking about using blockstates to store block data, not about misusing them to have 16 different blocks share the same ID. (Although I'm sure it'd be a 30-minute-job to make a "superItemBlock" that can place any number of blocks based on item nbt. Please don't! Having to deal with vanilla nonsense like having one item for 2 different blocks (e.g. torch) will be painful enough.)

[LexManos]

Basically, ids are assigned on the block state now instead of the block. So there is no 16 alignment or anything like that as there is nothing to align to. Blocks are no longer stored in trhe world. BlockStates are.

[Alpvax]

So now we can move any simple additional data (such as which section of a 6-block door) to states instead of NBT? We don't have to worry about the number of properties at all anymore? Do we still have to handle converting properties to ints or is that done automagically?

[williewillus]

@Alpvax

1. yes
2. yes, basically. combinatorial explosion can still be a problem so use a TE if you literally need a billion different possible states
3. there is no "conversion", the property-value map is saved directly as-is

anyways this thread is getting a bit off topic :P

[Alpvax]

Thanks very much.
Apologies, I believe that's my fault.
Are the vanilla tags becoming a complete replacement for the ingredient lists (At least in vanilla recipes)? Will forge continue to support recipes the same way or will using tags be recommended in preference to using the lists?

[liach]

@LexManos Just curious, how does minecraft distinguish between saved and unsaved block states? You know some blocks states may only be used in getActualState.

[LexManos]

It doesn't anymore that's the point.

[HenryLoenwind]

ouch. now I really hope that doesn't give combinatoric problems. because, tbh, our connecting blocks are quite spammy with blockstates...

re: off-topic: I wouldn't consider this too OT, if we want to talk about tags, we need to understand blockstates. So: are tags on blocks or blockstates? (just to get back on topic)

[Boundarybreaker]

Tags in this context are for Items and ItemBlocks; they're pretty much exclusively for crafting. As Asie put it, they're "essentially the oredict but Mojang".

There are tags for blocks, but those are for testing blocks in world. Block tags might be helpful for multiblocks, but this is about oredict at the moment.

I still really love the 12w49a changelog saying "Recipes can now refer to a tag instead of an item. This should save a few hundred thousand lines of code, somewhere."

[williewillus]

tags are on Block and Item instances. if you ever think you might need to tag a state separately, it probably should be split to a different Block ID.

as for the combinatoric problems, if it's working fine today then it should still work, the total available number of state ID's (integer max) hasn't changed

[LexManos]

We will have to fully investigate the I'd switch, I fully expect there to still be a expanded state for rendering things. But when it comes to any logic, or collisions, it should be standard block states. Before anyone explodes about "muh trillion blocksteatehs!" Please sit down and actually spec out what you're trying to do and why. And we can address it but we will NOT tolorate irrational fear mongering

[liach]

When I try to port the 1.7.10 getIcon to getActualState, I find that getIcon sometimes creates too many possibilities that I need to create 15625 states for rendering (luckily, they are not saved to chunks). In this case, should I switch to extended states or should I still use regular block states? I fear that regular block states may be saved to disk and slow down the game a lot.

[williewillus]

Anyways, what we wanna figure out in this ticket is the conventions under which modders will use the vanilla tag system.

Here's a list of conventions to think about:

• Snake case names for all tags, vanilla style. Not camel names like oredict (this is enforced by ResourceLocations being lowercase alphanumeric anyway)
• Ores and all their forms will use a special reserved domain ore. Like oredict names, the "form" will come before the "material". E.g. ore:ingot_iron, ore:block_tin, ore:gem_diamond, ore:plate_brass
• Other miscellaneous classifications of items modders might want to contribute to (e.g. a list of everything that can give a crafting interface, a list of all bowl-like items, a list of all bucket-like items) will use the special reserved domain common. Lists of this purpose will have plural names, like vanilla. E.g. common:crafting_tables, common:bowls, common:buckets
• If vanilla has a tag for something (e.g. buttons, doors), ignore all above rules and use that tag. E.g. minecraft:buttons, minecraft:doors

Any suggestions? Is there a point to having two separate reserved domains? Should we use the forge domain instead?

[Prospector]

Form going before material looks straight up wrong in snake case. And I don't think tags should be plural, a block isn't ever a crafting_tables, it's a crafting_table

[mezz]

For list_of_stuff the vanilla tag system looks like a great replacement for most of the ore dictionary.
For just the form_material examples given, I think the unifier (https://github.com/MinecraftForge/MinecraftForge/pull/4457) should be used instead.
I think together, these two things can replace the ore dictionary entirely.

[williewillus]

Form going before material looks straight up wrong in snake case.

Understandable, ID's are material_form so tags can follow that

And I don't think tags should be plural, a block isn't ever a crafting_tables, it's a crafting_table

That makes no sense, the entire point of tags is to group multiple different blocks together under a common identifier. Besides, Mojang themselves use plurals so I think we should follow (buttons, doors, saplings, logs, planks)

[Gaelan]

Unifier can't replace oredict/tags altogether, even for traditional "oredict" stuff, because there are often "replacement" items that need to be distinct for lore or gameplay reasons. For example, AE2 Pure Nether Quartz Crystals are gemQuartz, but need to be a distinct item so players can't double them again. Unifier is a great idea, but we still need the "distinct but equivalent item" functionality, so we can't get rid of oredict entirely.

[Boundarybreaker]

Mezz was saying unifier and tags together can replace oredict altogether. Tags can be used for distinct-but-equivalent cases like the gemQuartz case, colors of wool, or Chisel materials, and unifier can be for identical materials like metals.

[Prospector]

That's precisely what tags are for. Traditional ore dict moves to tags, and the old misusage of the ore dict is what the unifier is for.

[Gaelan]

Yeah—I guess my point is that anything in the unifier should be a tag as well, so that equivalent things are possible. Think we're on the same page.

[Alpvax]

I'm not sure "anything in the unifier should be a tag". But we definitely need to be able to tag unified instances. I can see automatically adding tags to unified causing problems (unless it is something like forge_unified:<material>_<form>, which could maybe make it easier to use in recipes).

Are tags accessible/modifiable in code, or do they have to be set using json (at least in vanilla)? Do we want to be able to modify tags in-code (I vote no, because I don't think overriding user's json modifications is a good idea, but I can also see that it could be useful in some situations).

[HenryLoenwind]

What Gaelan wanted to say is: If two unified items have different tags, it will lead to problems. Example:

Item A: foomod:shiny_copper tag: ore:ingot_copper, unified: ingot, copper
Item B: barmod:pretty_copper tag: barmod:metal, unified: ingot, copper
Recipe: 2 ore:ingot_copper + 1 ore:ingot_tin = 4 ore:ingot_brass

Now item B wins the unification. Um, what happens if someone wants to craft brass?

So I also think that unifying items must enforce that their tags are in sync.

[Alpvax]

I believe Items can have multiple tags. You can tag barmod:pretty_copper as ore:ingot_copper as well.

But +1 for unified tags syncing up.

[HenryLoenwind]

yes, but modders make mistakes. And this is the kind where the wrong mod (A) will be blamed for it. I hate those because Ender IO always seems to be that "mod A"...

[Alpvax]

But in the case that mod A is using the unification system, should it not be using the unified instance in the recipe?
Comments from #4457:

So by default, if modA:copper_ingot is registered to the oredict, and modB:copper_ingot isn't, but both are registered through the unifier, what happens regarding recipes? (Obviously modA should have registered the unified to the oredict, but what if they didn't?)

@Alpvax if you oredict before unifying, you will have a bug if your item is not the one chosen by the unifier. Unify first and then oredict the unified thing and there is no issue.

My understanding is that the same thing would apply here, if you are unifying, then you should tag the unified instance. That would make it a mod A issue.

[HenryLoenwind]

and how do you recommend to get the unified instance into the recipe json?

[Boundarybreaker]

Since tags are namespaced, there could be unified:copper_ingot or a special tag for unified items.

[Alpvax]

I believe that issue is covered in the unified PR is it not?

[Boundarybreaker]

Welp. Looks like 1.13 is delayed. Mojang's having trouble with their worldgen rebuild, so they're pushing 1.13 back and focusing on new content, so the 1.14 Update Aquatic features are being merged into 1.13. Doesn't look like we'll have to worry about this for at least a little while.

[williewillus]

We should work on getting the unifier polished and potentially into 1.12 in that case

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in a long time. If this issue is still relevant and should remain open, please reply with a short explanation (e.g. "I have checked the code and this issue is still relevant because ___." or "Here's a screenshot of this issue on the latest version"). Thank you for your contributions!

[SkyHawkB]

This is still relevant because this hasn't been decided on yet, and is a thing that Forge 1.13 should consider.

[liach]

Oredict is planned to be removed as said in the parley.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in a long time. If this issue is still relevant and should remain open, please reply with a short explanation (e.g. "I have checked the code and this issue is still relevant because ___." or "Here's a screenshot of this issue on the latest version"). Thank you for your contributions!

[liach]

Now, forge has tags like #forge:ingot/iron in place of old ingotIron. Should be resolved.