minetest 🚀 - Different "Dimensions" / Other worlds

I too favour the additional coordinate way, but its very hard to implement it in all parts of the minetest api.

@paramat proposes a very light alternative, where the realms are stacked in y direction.

est31 on 10 Aug 2016

There has been some discussion about inter-world movement in the forum and in IRC dev channel, over the last few years.

paramat on 10 Aug 2016

😕1

I have mentioned before about mapgen layering where different worlds can be assigned different Y levels within the -30,000 to 30,000 space and things like Nether, Aether, Moon, Planets etc all have their own space in the stack and is fully backwards compatible with no need to change set/getpos... The minetest engine itself will deal with displaying ONLY that layer and allow players to teleport between them as if it's another world.

tenplus1 on 10 Aug 2016

👍1

The altitude check for players in order to set skybox and gravity is quite lightweight as it does not need to be done often, once every few seconds is enough so it can be once per 100 globalsteps or so, as in my moonrealm mod.

I'm not opposed at all to the dimensional approach, but our vertical height is so big there is a lot of room for several stacked realms (reachable by vertical travel), plus a few more sealed off near the top and base only reachable by portal.

paramat on 10 Aug 2016

😕1 👎1

@tenplus1 I was aware of the ability to assign different levels of the world to different realms, however if you are a player who likes to dig through enourmous amounts of stone, you might find that the moon isn't as deep as the Earth. It would be great to have multiple sub-worlds with just as much space as the current Minetest world.

Besides all that, you also have mod support to worry about. Would we have to keep a list of dimension heights for everyone to use?

@paramat

but our vertical height is so big there is a lot of room for several stacked realms

That's true, but every once in a while, you get a player who gets a little too curious (and has a bit too much time) for their own good. The kind of people who adventure out to the borders of the world, for example.

C1ffisme on 10 Aug 2016

👍3

I suspect layers will add more problems than they solve. There will always be a limited number of them, and as digging a few 1000 deep is not uncommon, they'd have to be reasonably thick, limiting their number. Then, there'd have to be an allocation mechanism, whereby mods can reserve a layer, or part of it. Those different reserved areas would have to be enforced by the engine, to prevent mods (accidentally) stepping on each other's toes. The mods themselves would have to be aware of which layer they are operating in, and what its 'ground-level' is (many of them contain y-coordinate dependencies). There'd also have to be a way to configure different mapgens for different regions, again with vertical limit awareness. And there are bound to arise more issues.

Can't this be implemented as some kind of mulit-server support, where a player can transfer, or be transferred from one server to another ? That would probably be the solution that causes the fewest problems in the long run, and it avoids the issue of having to add realm- or layer-awareness to the entire minetest core. Every dimension / realm would have its own server thread in the same process (as I assume that's easiest to implement). Eventually, this could even lead to multi-server setups using multiple processes, which is probably more complex, as the client will have to set up a new connection, and the servers will have to contact each other and communicate about the player that's being transferred.

Other advantages:

Every realm can have its own mapgen
Every realm can have its own selection of mods

Rogier-5 on 10 Aug 2016

👍10 👎1

I, too, would favor a fourth coordinate, but for a different reason. One of the biggest planned features are Voxel Area Entities, right? The most straight-forward way I can think of to handle these is with a fourth, 16 bit unsigned integer coordinate. That would allow for 64k different active VAEs (and "64k should be enough for everyone" :-P).

What we call the "world" or "overworld" would simply be one of these (say number 0 by default). All the others would _also_ consist of a number of blocks of 16x16x16 nodes each.

With such a system in place, it's a simple step to mark one or more of these VAEs as "top-level worlds", and have a 4th coordinate on the player denoting which top-level world they are in.

As for the multi-server support, it should be possible to hack something together right now, where several servers share a user database, and only allow you to be logged in to one of these at a time (the others just forbid logging in via lua, there's hooks for that). Or if you logged out in a special portal space thingy, you can choose which server to log into. I wonder if someone has tried something like this yet…

bendeutsch on 10 Aug 2016

👍5

I'm also against a layer implementation of dimensions, for the previously-stated reasons that it will complicate the way that mods can use y-values (esp. for map generation stuff), and that it places a limit on the number of dimensions. It would definitely rule out any mod like Minecraft's Mystcraft, which allows players to create new dimensions through normal gameplay.

There isn't a particular implementation I'm in favor of, as long as each dimension contains a full map. The coordinate-based one could later make VAEs easier to implement, while the server thread implementation could make it easier to implement server transfers. The second one's justification might make more sense conceptually, though, since transferring players between dimensions could be similar to transfer between servers. VAEs aren't conceptually related to a separate coordinate.

EDIT: Actually, I take that back. I prefer the coordinate approach because the lua environments are guaranteed to be shared. Different server threads would presumably have separate environments, so you would have to take special measures to transfer mod data between them. If there is a good way to share arbitrary lua values, including functions/coroutines/userdata, I would be fine with the server threads approach.

raymoo on 11 Aug 2016

👍2

@Rogier-5 Can't this be implemented as some kind of mulit-server support, where a player can transfer, or be transferred from one server to another ?

I'd prefer between sub-worlds, rather than servers, to make sure that players can still use dimension mods in singleplayer.

C1ffisme on 11 Aug 2016

I'd prefer a forth coordinate, although it would be very hard to implement in a backwards compatible way

rubenwardy on 11 Aug 2016

I'd prefer between sub-worlds, rather than servers, to make sure that players can still use dimension mods in singleplayer.

@C1ffisme In singleplayer mode, even though there is no separate server process, there still is a separate server _thread_ within the minetest process. For multiple dimensions/realms, there would simply be multiple server threads within the minetest process. Initially, even the multiplayer setup would probably not have separate server processes, but separate server threads inside one process. As that's probably easier to implement...

Rogier-5 on 11 Aug 2016

👍2

I still support the "coordinate" way, but I object a little to naming it a "coordinate", since dimensions don't have any inherent spatial relation to each other.

EDIT: For backwards compatibility, I support to just make a decent effort to make API behavior similar to before, and abandon compatibility of dimensions with old clients altogether. I think this feature would be worth breaking backwards compatibility for, but it might be wise to bundle it with other backwards-incompatible changes release-wise, if there are any.

raymoo on 11 Aug 2016

👍2

@Rogier-5 I suppose that's also true.

@raymoo

I still support the "coordinate" way, but I object a little to naming it a "coordinate", since dimensions don't have any inherent spatial relation to each other.

Yeah, I don't know why I called it a coordinate, it would be more like a value attached to the player which specifies which dimension he/she is in.

I think this feature would be worth breaking backwards compatibility for,

Yeah. Probably. :wink:

C1ffisme on 12 Aug 2016

Anyway, we don't have to choose between the 2 approaches, we could have both, stacking realms allows conventional vertical travel between them which will be fun.

paramat on 12 Aug 2016

👎1 👍1

@paramat

stacking realms allows conventional vertical travel between them which will be fun.

Fun, but doesn't always fit the storyline or theme of the sub-game. It wouldn't make sense to nerd-pole even 30 kilometers to get to the moon, because outer space starts at 420 kilometers. (At least, that's where the heterosphere ends.)

C1ffisme on 12 Aug 2016

👍1

It makes sense for things like cave realms, though.

raymoo on 12 Aug 2016

Fun, but doesn't always fit the storyline or theme of the sub-game

Just because it doesn't make sense for one doesn't mean you should disallow it for all

rubenwardy on 12 Aug 2016

Having separate maps in the same server thread would also allow for closer coupling between dimensions, like a shadow world in which anything built in the overworld immediately gets a matching node placement.

As for VAEs: it might also make sense to start with multiple numbered maps, and then VAEs could get their node data from a given map number (which would naturally correspond to a different dimension, but this would not matter because players could never enter it).

bendeutsch on 13 Aug 2016

🎉1

Backwards compatibility: for each function which takes 3D coordinates, create another one which takes 4D coordinates (getMapNode → getMapNode4D). Then you switch everything to 4D coordinates internally, and the 3D ones become wrappers where the 4th coordinate (what I would still call it) is always 0. Time passes, and clients get updated. All function calls with 3D coordinates get replaced by the 4D ones.

Then drop support for the 3D ones entirely. Their names then become free, and become aliases of the 4D ones. More time passes, mods and callers update to the original names again. Then drop the 4D ones → migration complete.

bendeutsch on 13 Aug 2016

Then you switch everything to 4D coordinates internally, and the 3D ones become wrappers where the 4th coordinate (what I would still call it) is always 0.

When you are in a different position in the 4th dimension (let's say "w = 1") and a mod uses a set_node that always defaults to 0, the node will be changed in the position "w = 0 " instead of where you are. And you can't directly assume the action of a mod is always tied to a player, so this won't be solved using the same "w" coordinate as the player by default. The mod would have to be updated.

Mods that use relative coordinates and that don't create position objects on their own will be fine (actually.. that's probably the case for most mods).. but otherwise they'll have to be changed or they will only work in w=0.

Doing things like this would make your mod lose the 4th coordinate data:

a = { x = b.x * 2,  y = b.y * 2,  z = b.z * 2 }

Mods should give more use to those convenient vector.* functions.

@raymoo:

I object a little to naming it a "coordinate", since dimensions don't have any inherent spatial relation to each other.

Coordinate axes are, by definition, spatially independent of each other (otherwise a coordinate point wouldn't really be unique for each point of space). A 4th dimension would have the same level of spatial relation as the other 3. Whether you can move around or perceive that 4th spatial dimension is a different thing.

Ferk on 13 Aug 2016

The funny fact is that extra dimensions won't cause a much higher data consumption than stacked realms in a single 31km^3 world. Both allow to explore the surface and dig down to earn ores.

SmallJoker on 13 Aug 2016

I don't think the problem is data consumption, I guess the biggest problem is how to store that data.
I assume there's a limit in the size of the variable that stores the position of a node.

Stacked realms don't need to increase the size of that variable. A 4th dimension would require a different approach, no idea if there's an appropriate way to do it.

Even when celeron55 was planning Voxel Area Entities he was thinking to use up the space from the unused regions of the world to store that data, instead of a completely separate instance.

Ferk on 13 Aug 2016

Yeah, I have to admit reading all the work on VAEs is still on my TODO list… hopefully soon™.

One problem with calling the dimension identifier a "coordinate" is that non-integer dimensions make no sense. The w value would always need to be handled seperately. Directions and distances between dimensions, perfectly calculable in ℝ⁴, would need to return "meaningless" with dimensions (incidently, with stacks in a single map, which I call "strata" if we're looking for a term for them, directions and distances would seem to return meaningful results even when they should return "meaningless". Just throwing that out there).

As for my "4D wraps 3D" approach: that's a long-term approach. During the first phase, all mods and server code would gradually become dimension-aware, but all servers would only have one dimension, so old mods will still work while all code is updated. The first servers to have more than one dimension will need to have _only_ dimension-aware mods. And pre-dimension worlds may need explicit converting.

bendeutsch on 13 Aug 2016

Non-integer dimensions don't make that much sense in Minetest in general. A lot of 3D values for {x,y,z} are meaningless when it's about indicating the position of a node. I assume the position internal variable we are talking about is already using integers, otherwise it would be a lot of wasted storage, even in the current 3D world.

Maybe for entities you can have fractions, but I doubt entities would be a problem.

As for the "4D wraps 3D" approach. Yes, as long as mods update it'll always be ok in the long term. I just wanted to point out that this wouldn't make old mods compatible with 4D worlds.

Ferk on 13 Aug 2016

Just read the VAE docs from the forum roadmap. If we go with the general idea that VAE node data is normal map node data from "somewhere", which I'd also say is the most logical path, then it's not relevant whether that data is in another stratum or another dimension. So we can keep VAEs out of it.

My vote is still on 4D coordinates, i.e. each world has up to 2¹⁶ maps. Leave strata for things you could physically climb / dig to; there's still plenty of use for that. And enforcing stratum boundaries in a 3D map is probably about the same amount of work.

bendeutsch on 13 Aug 2016

Please somebody mark this as WONTFIX, we already have 3 (!) dimensions, they are called “X”, “Y” and “Z” and they are part of Minetest since like … forever? Stupidest feature request ever. :P

Haha, jk, of course I understand your feature request, but, I am strongly opposed to call this feature “dimensions”, this is just stupid Minecraft speech and doesn't make sense. We shouldn't adopt it.

I prefer to use the term “parallel worlds” or maybe even “parallel universes”. See also: https://en.wikipedia.org/wiki/Multiverse
https://en.wikipedia.org/wiki/Dimension

Wuzzy2 on 13 Aug 2016

😄2 👎1

You're looking for a technical term, right? Whether a game refers to them as "dimensions" (sci-fi, Discworld too) or "planes" (e.g. Dungeons and Dragons) is up to the author.

Technically, I'd call a "world" everything that is under the "world" folder. Something each server has one of; something a singleplayer chooses from the main menu. Currently a world has one map, one list of players, one "global chat", one list of entities and so on. With this feature, a world would have several "subworlds" with their own map and their own list of entities, but there would still only be one list of players and one shared chat. I think "parallel world" or "universe" would not fit the bill. Maybe "spaces"?

bendeutsch on 13 Aug 2016

@Ferk I'm not saying that a dimension "coordinate" would be independent from other axes. I'm saying that it doesn't make sense as a spatial coordinate, because (even if they were numerical coordinates), there is no inherent (non mod-enforced) difference between the relation of dimension 2 and dimension 4, or dimension 2 and dimension 1000. All other axes do have some relation. For example, if object A is at (0, 0, 0), object B at (0, 2, 0), and object C at (0, 0, 10000), then A is much closer to B than to C. But the "distance" between any two objects in different dimensions is the same, because if two positions are in different dimensions, they are not comparable except to say that they are not in the same dimension.

@Wuzzy2 @bendeutsch I like "planes"

raymoo on 13 Aug 2016

👍1

4D node co-ordinates are overcomplex and unnecessary, as is a strict adding of a 4th spacial dimension.

paramat on 14 Aug 2016

👍1

@Wuzzy2 The truth is, they aren't always different planes of reality. Sometimes they are a different time period, sometimes they are different planets, and sometimes they are simply private territories given to players for their own private use.

The real term should be "Sub-Worlds". This makes since, because the world right now is an area that is 62^3 kilometers big. So having multiple worlds inside of one big "world collection" that we call a world, it makes since to call the smaller ones "sub-worlds".

C1ffisme on 15 Aug 2016

Something that hasn't been mentioned yet is the need to be able to add dimensions during gameplay, rather than just register them at mod initialization. A couple examples of things that need it would be a Mystcraft clone or some personal pocket dimension mod.

raymoo on 11 Sep 2016

👍4

@raymoo That's a good point.

The only thing weird about that, is that you can make something as big as another dimension, but not something as small as a new item or node type while playing the game. :stuck_out_tongue:

C1ffisme on 11 Sep 2016

😕1

Well, you'd somehow need to persist the newly-made thing, which I would guess dimensions would get automatically from being part of the map, while items only exist once they're registered at the beginning. Though that does give some ideas of hacky stuff you could currently do in a mod, like let players create entirely new items that get introduced on the next restart. I can't think of any use case that would need registering items or nodes on the fly to be done nicely, though. Items are pretty flexible in what they can do, and all their behavior can come from the metadata. The only thing missing right now is changing client-side things like the description, texture, tool groups and things. Nodes require more since they have various effects by existing in the world, but it's basically the same issue.

The way I see it is that player wouldn't be creating any arbitrary kind of item, but specific kinds of items that the mod allows, which could be done with item callbacks + metadata. The same thing applies to nodes. We can extend this patterns to dimensions: players will only be creating certain kinds of dimensions. In Mystcraft these are Myst book world things, and in a pocket dimension mod it would be pocket dimensions. So creating a new dimension is more like creating a new itemstack of an existing item, possibly with customized metadata, rather than like registering a new item.

raymoo on 11 Sep 2016

I'd much rather prefer to see a multi-server approach to this problem, where players log in/out of servers in a network and get redirected to other ones as needed.

The in-game logic is vastly more simple this way - redirection is done at the session level and doesn't need any specific mod handling other than triggering transfers, mapgens are always exclusive and can avoid the edge problems, and server operators can choose to use different inventories for players (e.g. on the lobby server there's nothing in your inventory, the survival domain always shows your survival inventory, currency goes with you to all domains etc.).

Lastly this approach would make implementation almost not require any protocol changes - there would just be a new packet type from the server to the client (client server transfer request).

A properly crafted mod or website could even create additional realms/domains on the fly. Combine this with socket-activated servers that shutdown automatically when the last player leaves and you have an infinitely extensible world that uses zero resources for realms which are unoccupied.

sofar on 12 Sep 2016

👍3 👎1

@sofar This has been mentioned in the conversation before, but it would need a lot of specific mod handling. If mods keep track of player state or global state (persistent or not) of some kind, which would need to be shared between servers.

If some of player state is functions, such as with minetest.after timers, there is no easy way to transfer the data because functions are not in general serializable, because of upvalues that may be userdata.

Mod global state across different servers in general opens a big can of worms called concurrency. Any global state at all would need to implement some kind of way for each server to modify and view the same state. Concurrent code is prone to bugs, even more so when the processes wouldn't even have the same lua environment, and it would make modding much more difficult when the mods involve any kind of state that could in any way affect a player, if the mod wants to support multiple dimensions.

raymoo on 12 Sep 2016

👍1

Mods will need to store data in player metadata, that's all. I see very little reason to pose that this is even required - the problem you describe seems minor at best. If a player transfers from world A to world B, and he retains their inventory and basic player stats (_if_ that's even needed or wanted), then for all intents and purposes the player is _offline_ for world A, and nothing in that world should affect player A.

If two servers share the players metadata folder, the only problematic thing currently would be the player position (and a mod could handle this). All other data can be shared between two concurrently running servers (while there may be edge conditions, I don't see any issues in resolving them, e.g. a player attempting to log into two realms at the same time).

But

On a larger scale, besides being solvable, it will likely not even be _needed_ in many cases. There are hundreds of bungee MC servers where this mechanism is used to transfer players from a "hub" portal to various "mini games" and none of the player metadata is transferred between the sub-servers.

So logically, we can do this in several _small_ and _incremental_ steps:

1) enable server redirection
2) allow player metadata storage sharing and/or transfers, work out the kinks

Then, after (1) is complete, and this is technically a minor change, the game server ecosystem can explore and develop the possibilities of the mechanism, and I'm pretty sure we can quickly solve the problems posted by (2).

I don't think your "mod global state" problem is real, but, if any data needs to go from one server to another with a player, it can be handled by embedding the metadata in the player metadata, and proper save/restore code. Again I think it's great if that could be added, but the server redirection mechanism enables uses that don't even have that problem to begin with.

sofar on 12 Sep 2016

Lastly this approach would make implementation almost not require any protocol changes - there would just be a new packet type from the server to the client (client server transfer request).

IMO there'd need to be a way for destination servers to know whether the transfer of a player is allowed. The best (but not the easiest) way would probably be a server-server protocol. Which I suspect will eventually be required anyway.

1) enable server redirection
2) allow player metadata storage sharing and/or transfers, work out the kinks

3) allow active objects in general to travel from one server to another. Either by themselves, or in the company of a player (e.g. a pet, an attached object, etc.).

A properly crafted mod or website could even create additional realms/domains on the fly.

A meta-process that manages different minetest servers could be useful. When connecting, the player logs in to the meta-process, which redirects the player to the server he last visited, or to another server if that is more appropriate. When neccessary, the meta-process could start a new minetest server process.

All of this (meta-thread, multiple server threads, and of course the client thread) might also be combined into a single process, for smaller worlds or singleplayer worlds.

mod global state

IMO instead of having a global mod state, a message passing mechanism could be used between mods on different servers. IMO that should be sufficient for most purposes.

Rogier-5 on 12 Sep 2016

I thought about it and multi-server could be used for the use cases I had in mind, as long as they had additional features. Rogier did mention some things that would help, but I think I should give some examples of features that would not work with just shared player metadata. A couple examples will be from existing mods and the rest will be either from hypothetical mod ideas I've had or features I have thought about putting in a mod I am working on right now.

The first example is the factions mod. Factions can hold power, own land parcels, and have player members. If two dimensions are conceptually part of the same world or universe or whatever, it would make sense that the faction will carry over across those dimensions, and can share the same power, using it for land claims in them. It should also have the same members in either side. Player metadata could be used to make it easy to access a single player's membership, but that is not enough. Faction power or land use is obviously not tied to a player, and will be shared global state between the dimensions, which will need to be handled safely in the face of possible concurrent accesses. There is also the fact that it needs to be possible to get the list of members of a faction. The only easy way to do this with only player metadata shared is to iterate through all player metadatas and see which players are in the faction, which might be expensive to do too often on a large server. The issues with this would be fixed by some way to manage global state, whether it is with a shared store or with message passing as Rogier suggested. In either case, there would need to be some way to work transactionally on the global state, otherwise different servers could mess up the state. With a shared data store, you could use locks. With message passing, it's a bit more complicated, but you can build a data store with locking on top of message passing also. Hopefully modders will not have to implement locking every time they want to share mod data.

Another example is one of those improved chat mods I am having trouble finding. Message passing would definitely be an easy solution for passing chat messages between servers, though things like password-locked channels would again require some way to communicate shared state safely.

A feature I am thinking of including in a magic mod is the ability to make a bond with another player that allows targeting them with a spell cheaply, regardless of their location. For actually casting a spell, just message-passing would suffice. The feature would impose a limit of one bond per player though, so if someone bonds with a new player, their old bond will be removed. This will either require editing the metadata of both players, or keeping some global data separate from player metadata. Assuming the first way, since the earlier example already covers data sharing, the feature would need to edit the metadata of players that are not online. This means the player metadata store will need locks, because of the following example. Player A and B are bonded with each other, and are currently playing on different servers. At around the same time, player A and B bond with players C and D, respectively. The actions taken by the server player A (respectively B) is on are

Remove player B (A)'s bond with A (B)
Set player A (B)'s bond to C (D).
Set player C (D)'s bond to A (B).

The two transactions need to be performed sequentially, or else the data will potentially be munged (either or both of players A and B end up bonded to nobody, while C or D is bonded anyway, resulting in an inconsistent state of one-way bonds).

Another feature that might go in the same mod is teleporting to player-built structures. To teleport to a location you would need to be able to check that the location still has a valid teleport gate or whatever. With dimensions in the same server, you could just load and check the nodes, but with separate servers, you will need to make some mechanism to record valid structures and share the state. This is really more of the same, fitting in with the earlier examples that require data sharing, but with a shared map environment you wouldn't even need to keep the state in the first place.

A hypothetical mod idea I have had is one where you commune with spirits and try to get them to do things for you. Spirits would be nonphysical beings (not represented by a lua entity), though they might spawn avatars in order to carry out some task. The same spirit could also commune with multiple players, e.g. a false god that does things in return for worship. For balance reasons, it makes sense to limit the number of tasks a spirit can carry out simultaneously, and thematically it makes sense for spirits to be more reluctant to take malevolent acts against people it likes. This is just more shared state.

So my new position is that it is possible to have a modding experience close to that of a shared environment, even with a multiple-servers implementation of dimensions, provided that these additional features are provided:

Message passing
The player metadata store can be modified transactionally
An implementation of either state synchronization (with good consistency guarantees and transactions) or a global data store with transactions is provided. Either could be implemented using message passing, but ideally in core lua. If for some reason this isn't wanted, somebody could implement it as a mod. Hopefully that person writes good code and is wary of concurrency problems.
- A player metadata store would automatically give you a global metadata store, just use keys that aren't valid player names
- A global data store should allow transient values, because not all mod state needs to or makes sense to be persistent.
- A normal DBMS could also be used, as has been suggested for mod data in other issues.

I still think separate servers is a worse way to implement dimensions. Modders will have to learn and understand these abstractions, and they will add overhead to writing code in general. I'm not strongly against it anymore if these additional features are implemented, though, because I can see now that working around multiple-servers dimensions would be manageable. However, since getting multiple-servers dimensions on par with single-server dimensions would require a lot of extra features, it may be more work to implement.

raymoo on 13 Sep 2016

@raymoo Good to have some examples. That makes the problem less abstract.

Shared storage may best be implemented using remote procedure calls to a single dedicated shared storage thread; for improved concurrency, there could be multiple such threads, up to one per mod. A transaction is embodied by a function call in a natural way. That means much less risk of hard-to-diagnose locking problems. The average mod writer should be expected not to be sufficiently familiar with the complexities of concurrent programming, so IMO, locks, transactions, etc. should be hidden behind a higher-level abstraction as much as possible. In order to avoid distributed deadlocks, RPCs should only be allowed to the central storage thread(*). Communication between other threads, or from the central storage thread to any server thread should always use message-passing.

Of course, even with RPCs and messages, mod writers can still create race conditions. They might even implement their own locking scheme, and most likely get it wrong...

(*) Alternatively, any function servicing an RPC could be prohibited from initiating another RPC

I still think separate servers is a worse way to implement dimensions. Modders will have to learn and understand these abstractions, and they will add overhead to writing code in general.

Whatever method is used, whether it be levels, an additional dimension (i.e. coordinate), or separate servers (which can still exist inside a single process!), modders _will_ have to learn new abstractions. The question is _who_ needs to learn them, and _when_ does it require additional coding (i.e. overhead).

In this context, I think we can distinguish basically two kinds of mods: those that handle player data (experience, hunger, inventory, factions, etc.) and those that handle in-world data (ores, and node definitions in general, MOBs, ABMs, etc.).

Shared player data will be easier to handle if realms are implemented in a single server (using layers, or an additional dimension), as data is naturally shared. OTOH, in-world data will be easier to handle using a multi-server solution, as that allows mods to be oblivious to the multi-server case, unless they specifically want to make use of it.

In the single-server case, may mods _will_ need to be made aware of the multiple realms:
It should be possible to enable mods for certain realms, but not for others. In the additional-dimension case, all positions would also have to include the additional coordinate. In the case of layers, ore generation which is depth-dependent should be aware of the layers. MOBs that spawn underground should be aware of the layers, etc. etc.

So basically, IMO it boils down to:

whether sharing player data between realms should require additional work, or whether non-player-data mods should require extra work to be made aware of the existence of multiple realms
whether the consequence of using mods that are still unaware of multiple realms will be an inconvenience, a nuisance, or a fatal problem.

An additional consideration is that multiple servers allow the load to be distributed over multiple machines in a seamless fashion. The single server solution does not, and instead adds additional load to existing bottlenecks (e.g. the single-threaded lua implementation), making it increasingly difficult to improve the situation in the future.

Rogier-5 on 13 Sep 2016

Ok, I agree that RPCs on top of storage threads would cover all my use cases. The only thing that I can come up with that feels a bit inadequate is that you couldn't abstract similar calls into ones that take functions or something like that, and that's just a code niceness thing. Though you could hack up your own closures to get around that.

I also agree that the changes to positions would be a burden, and for me that is the strongest argument for a multi-server implementation of dimensions. I support a multi-server implementation of dimensions at this point.

I'm unsure of having different mods enabled between different dimensions. You could end up with unknown nodes and items if player inventories are transfered between dimensions with different mods enabled. In the basic dimensions use case described in the OP, player data and mods is meant to be shared. Maybe there should be some way to group dimensions together as cohesive units? Global mod data could be separate for each group, so that going between these units would be more like the "server transfer" use case where the different dimensions really are meant to feel like separate servers. There could also be some kind of gating that causes transfers to fail if the receiving server doesn't have all of the player's items registered. Still, there would be a danger of messing up item metadata if a mod in one server overrides the behavior of an item that isn't overridden in another server. I would guess that item metadata handling is usually (? or should be) done robustly, so that might not be a big problem.

What drove my original arguments is that while the dimensions described in the OP can be considered a subset of server transfers, it is still distinct enough from server transfers that it would benefit from special handling, so that they aren't only limited to what can be done safely between any two unrelated server threads. Now I see that multiple server threads are a viable way to implement dimensions, but I still think Minetest would benefit from distinguishing between dimensions that are supposed to comprise a cohesive, mod-sharing world from dimensions that are more loosely related. This is also why I saw no performance problem with single-server dimensions: I don't think having 50 active blocks across 4 dimensions will be significantly more expensive than having 50 active blocks in a single map, and if you want server transfers on top of that, you can implement them separately. Considering cohesive groups of dimensions as distinct also makes registration of nodes and things not much of a problem in single-server, because you are sharing that data anyway. Mapgen stuff would need a bit of work, but would not need to be messy. API-wise, you would just need to make it possible to create mapgen configuration objects or something which have their own methods equivalent to the global registration functions (ores, decorations, etc.). You will have to deal with something similar in the multi-server implementation anyway, if you are planning on making it possible to make new dimensions on the fly, because you won't be able to just use a different mod for each server.

That's related to another reason to distinguish MC-like dimensions from server transfers. You want mods to be able to control dimensions, but you also want the server operator to be able to create worlds that are linked together. These are two use cases that deserve some separation.

raymoo on 13 Sep 2016

Oh yeah, another thing I forgot to mention on top of being able to make dimensions on the fly is needing to be able to customize new dimensions' mapgen parameters and registrations. This is necessary for something like Mystcraft, and maybe a pocket dimension mod too, depending on how ambitious the modder is.

raymoo on 13 Sep 2016

I'd much rather prefer to see a multi-server approach to this problem

@sofar How much loading time would one expect for a client, when moving through the realms, for loading additional assets, handshake etc.?
Is there a concept of preloading or keeping the mapchunk during loading active (as a waiting area), that might compensate?

t4im on 13 Sep 2016

@t4im

In a single-server, I'd assume all assets are always loaded upfront (and only the entire map is resent & redrawn)

In a multi-server setup, the client would have to reinitialize the entire session, so it would be like logging onto a new server (e.g. 10-60 seconds depending on assets).

I don't see a reason to keep mapblocks alive or cached client side, it complicates things and consumes memory for things that _may_ be useful.

sofar on 13 Sep 2016

In a multi-server setup, the client would have to reinitialize the entire session, so it would be like logging onto a new server (e.g. 10-60 seconds depending on assets).

There could be a gateway or front-end server (or thread). The client connects to the front-end, which sends all known assets to the client and forwards the connection to the appropriate realm server. When the player travels between realms, the front-end server forwards the connection to a different realm server instead, and optionally sends any additional assets that are needed. The front-end server could run as a thread in the client or in the server process, or be a separate process. This would also allow a player to travel between unrelated worlds owned and adminstered by different server admins, without the cooperation of those server admins, except that they'd need to install a mod which implements the portals. Of course, it would only be possible for the player to keep his inventory and other player-specific data when travelling between realms on cooperating servers (provided those realm servers _allow_ him to keep his data...).

Rogier-5 on 14 Sep 2016

My last longpost was a rambling 4am post so I thought I should add some things and summarize my position. This post will have some repeated points from my previous one.

I'm not sure anymore that the API burden imposed by single-server implementation has to be very high. The argument earlier was assuming a coordinate-based system, but having a shared lua environment doesn't mean a dimension axis has to be inserted into position vectors. Moving between dimensions could be a separate operation from moving to a position. There are still many other places where you would need to specify a dimension, but you could pass it separately when needed, since with explicit-transfer, many operations such as setting object position would just need a vector without dimension. Places where dimension inputs would need to be introduced include global calls that involve positions, like get_node or get_objects_inside_radius, and callbacks that receive position input, including pointed_things. Basically anywhere that already handles positions, except when the dimension is implicit, as in object methods.

I think it would be much easier for modders to adapt to this than to the methods of managing shared state discussed earlier. Fitting code that handles positions would just require the straightforward change of threading through dimension information when necessary. Compared to making sure your global state works across different server threads, the changes required to adapt to single-environment dimensions are more mechanical and harder to screw up.

Changes in mapgen-related API will be required in either single-server or multiple-server implementations of dimensions, simply because mods will have to be able to control the mapgen of new dimensions. For single-server, I would suggest having some sort of mapgen object associated with each dimension, which you can register ores, decorations, and other mapgen stuff on. Such an object might also be a parameter to any function that creates a new dimension. I'm not sure about multiple-server, but anything where the registered information is serializable could be done with RPCs. For both cases, on_generated callbacks might be more complicated to adapt, since you would want some way for mods to be able to decide which dimensions to operate the callbacks in. In the case of single-server with dimension arguments passed around, it could be as simple as the callback checking whether the current dimension is one it wants to operate in, but modders might want to be able to add mapgen features from other mods into their own dimensions, so there would need to be some way for other mods to register which dimensions to operate a callback in. This could be done in the mod itself, but it could help to have tools provided. Multiple-server would work similarly, except I would guess that instead of a dimension argument, you would have some global function or variable that provides the current dimension.

You mentioned possibly having RPCs to any server thread, and this will be required for dimensions to operate nicely on the world of another dimension and provide feedback back in the dimension that initiated the operation. Without RPCs to server threads, modders get to experience the fun of asynchronous programming via message passing. I don't think adding such RPCs will complicate the modding API much, but could add more work in implementation.

So based on these points, I think adapting mods to dimensions would be easier and simpler for modders in a dimensions API that features a shared lua environment. The only places that I can think of where single-environment loses is in backwards compatibility. While multiple environments would also be backwards incompatible since state wouldn't carry over across dimensions, single environment is potentially worse, since parts of the API involving position will need dimension included. This could possibly be alleviated by things like an implicit default dimension to fill in missing dimension arguments, but this would need to be done carefully since mods might sometimes call callbacks themselves (e.g. when emulating item_place/node_dig). It also couldn't really work for things like globalsteps where there is nowhere to get the dimension context from. Backwards compatibility is a weak reason to deny something like this though. Backwards incompatibility is a burden until the community has adapted, while a worse API is a burden for the entire time the API is in place. Dimensions can wait until whatever release allows breaking compatibility without a deprecation period.

The modding API is the part I wanted to talk about most, since that was why I switched from single -> multiple environments originally, but I should look at some other areas that have been discussed.

One of the benefits of multiple environments (implemented with separate server threads) might be that it brings Minetest closer to an implementation of server transfers, or a "multiple servers in one" sort of setup on one server. First of all, those are separate use cases than the one originally introduced in this issue. Dimensions are part of a cohesive world, and are controlled by mods. They add additional layers (but not in the physical sense like discussed in #4503) to the map. They allow modders to extend the map with new places to go that are different from whatever is the "main" map (if the game has one at all). Server (or sub-server) transfers are different. They could have different mods enabled, which will probably be controlled by the server operator, but maybe there could be some mod API as well. Inventories may or may not transfer over, since items in one server might not exist in the other one. Obviously, a single-environment dimensions implementation wouldn't help with adding server transfers or subservers. But a multiple-environments implementation would still need significant work to implement both. Multiple environments is supposed to be better for giving the choice of whether to share data, but if they are used to implement dimensions, you will need more to support it. Some dimensions might be "just an extension of the world", while other ones might be representing "some other server". Moreover, if you really did have something like another server, you would want each subserver to have its own nether, own pocket dimensions, etc., as well as its own mod data. So a multiple-environments implementation that also wants to support something like subservers or server transfers will need to add some way to create dimension groups, with each getting its own global data threads, its mods being able to control dimensions within its own group, and so on. On the other hand, if you implement dimensions within a single environment, and then separately implement server transfers / subservers, you get a natural separation and won't have to deal with grouping subservers or managing those groups. You will still need message passing and data threads and things if you want to share data between them, though. In any case, I wouldn't sacrifice a better implementation of dimensions just to save work in implementing subservers later on.

Another point that has been discussed has been performance. The main reason that multiple servers would give a performance boost is because the load could be spread across multiple cores. I think this is a valid point, but that the performance increase wouldn't be that high in a game like minetest_game, or anything resembling Minecraft, since most players will probably spend most of their time in the overland map, putting most of the load on that thread anyway. It could see significant benefits on something with a more interesting dimension layout, such as a game or server where everyone decides to live in their own pocket dimension. This kind of setup is plausible considering that in the absence of protection, players will already travel far from spawn to avoid griefers. But my opinion is that a performance benefit that depends on player behavior to be substantial is not that compelling. If subservers are implemented separately from dimensions, those could be done with multiple threads and you would be able to use multiple cores there, where it's more likely that you would have players spread out across them.

@Rogier-5 brought up the concern that adding dimensions to the single-threaded implementation would slow down the game. I don't think such a performance hit would be substantial, since processing 2 active blocks around 20 players costs the same whether they're spread out across one map or twenty. You might have less processing to do if players are playing near each other, but if players want to be near each other, they'll do that regardless of the presence of multiple dimensions. Another performance sink could be that you would have another level of indirection to the map since you would have multiple dimensions. I think this could be fixed just by caching the dimension that's being currently processed (for ABMs, or entities, or whatever), since that is where most of the processing is going to be. Accesses across multiple dimensions would be slower very slightly, but they would also be slower if dimensions were implemented as server threads, since you would need to use RPCs or message passing, depending on what you are tryiing to do, which would involve (de)serialization and, in the case of RPCs, waiting for the other thread to process the request. I would guess that accesses across dimensions would be significantly slower with multiple environments than with a single environment.

tl;dr:

Dealing with dimensions with multiple environments would take significantly more effort than a single environment.
Subservers and dimensions as discussed in the OP are separate features and have a good chance of benefiting from separate implementations.
Performance benefits from multiple threads is dependent upon players spreading themselves across multiple dimensions.
Multiple dimensions with a single thread would not make the game significantly slower than it is now, all else kept equal.

raymoo on 20 Sep 2016

Forgot to mention another reason you might choose a server thread implementation, requiring a multiple-environment API. You would have to touch less of the existing code because you aren't making any changes to the map structure, so it might take less effort to implement. I would imagine that the effort required by single and multiple environment APIs would be within a magnitude of each other though, assuming in the multiple-environment case that you did everything you need to make it suitable for the "dimensions" use case described in this issue.

Also, a single server implementation might require no or minimal changes in the client and protocol, as long as what I'm going to describe is viable (I'm not sure since I haven't seen most parts of the code). Assuming dimension is handled specially compared to the other coordinates, dimension could be kept track of only on the server. The server would just send the client map data and updates for the dimension the player is currently in. If it doesn't already exist, you would want a new packet type that instructs the client to clear the map cache, or else traveling between two dimensions would result in the player seeing blocks from both dimensions spliced together.

EDIT: Added second paragraph.

raymoo on 22 Sep 2016

Something I'd like to point out is that having subworlds could also "fix" the issue of world size - Either could build it in that on world creation, one can set the max size of the world (e.g. put in 2x2 for 2 maps long by 2 maps wide) and have say, maps divided into 16ths where being 4 km from an edge prepares the next map for a seamless transition, meaning in a single player world at any time at most four maps are loaded) or make it possible for mods to load subworlds side by side

Kill two birds with one stone there (assuming I'm not horribly underestimating something that would make that go horribly wrong)

Icalasari on 30 Sep 2016

@Icalasari yeah that's my idea as well.

est31 on 30 Sep 2016

It's possible to stack the same character of landscape vertically to create more land area (without seamless edge-matching, which is very difficult), i could write a mapgen now to do this. However i don't find this idea very exciting and consider the one surface world big enough (horizontal world size is not a problem). Any stacked realms would be much more interesting if they have different characters (floatlands, moons, exoplanets, deep space, dimensions).

paramat on 30 Sep 2016

seamless edge-matching, which is very difficult

I've said it already, and I say it again: seamless edge matching is not a hard problem. It can be solved quite easily.

Perlin noise bases on normal pseudorandom grid nodes which it then interpolates. In the source code you can see it here. Any given output value of a 2d perlin noise only depends on those 4 corner points. For 3d perlin its similar, here you have 6 corner points.

All we need to do is to add a bounds check and make the outermost values use the same noise the other area uses.

This code shows how noise2d_gradient can be modified to seamlessly match its own edges (aka if you want the mapgen to repeat itself in each map).

    // Calculate the integer coordinates
    int x0 = myfloor(x);
    int y0 = myfloor(y);
    if (x0 <= -LIMIT) {
        x0 = LIMIT;
    }
    if (y0 <= -LIMIT) {
        y0 = LIMIT;
    }
    // Calculate the remaining part of the coordinates
    // [...]

But of course, more advanced methods are possible where different seeds are used in edge conditions, etc. Also, stuff like trees/dungeons/Caves will obviously be cut at the border, but it can be solved with similar methods as the problem is solved for mapchunks.

est31 on 30 Sep 2016

However i don't find this idea very exciting and consider the one surface world big enough (horizontal world size is not a problem)

It does still effectively cover that issue though for anybody who finds the surface world not big enough. Also, the vertical stacking is another thing that allows, building up into the sky til eventually you enter a literal asteroid map, or down so deep, past the initial map, that you start doubting there'll be anything until suddenly a massive cavern opens up, hundreds of kilometers down

If anything, it seems like if that implementation was included, it would be more work to go out of your way to not include the possibility for mods to also allow it to load along the x and z axis

Icalasari on 1 Oct 2016

Est31, yeah the basic blending of noises occured to me too, and is fairly simple, however i suspect many other issues will be difficult. It's not needed and not worth workiing on. If it could be contained within a single mapgen that might be ok, but it will require many changes throughout the engine to support it.

It does still effectively cover that issue though for anybody who finds the surface world not big enough

Our 2 mapgen devs, hmmmm / kwolekr and myself, find the idea of 'world not big enough' funny and ridiculous, and strongly oppose enlargement. A few players ask for it only because of Minecraft's size.
The most significant dimension of a world is the smallest dimension not the largest, so in fact Minetest is more 'spacious' due to it's vertical height. Horizontal distance tends to create more of the same whereas vertical distance tends to create new things and variety. We can enlarge the world by having a variety of stacked realms.

it seems like if that implementation was included, it would be more work to go out of your way to not include the possibility for mods to also allow it to load along the x and z axis

No, completely different.

paramat on 1 Oct 2016

The most significant dimension of a world is the smallest dimension not the largest, so in fact Minetest is more 'spacious' due to it's vertical height. Horizontal distance tends to create more of the same whereas vertical distance tends to create new things and variety. We can enlarge the world by having a variety of stacked realms.

And yet, it feels good to have a world where you can keep running for hours and still never hit an end. Also, it means certain biomes like Minecraft's "Mushroom Island" Biome can be rare, and using things like nether portals or minecarts to skip "boring" terrain becomes a good investment.

I can't help but feel cramped, if not claustrophobic while playing Minetest. It might be because my view distance is turned down low, (because I need to keep my laptop cool), but I feel like at least knowing that there is far more land to explore would help be more immersive.

But I suppose this is a bit offtopic, since this is a topic about dimensions, not Infinite Worlds.

C1ffisme on 2 Oct 2016

Our 2 mapgen devs, hmmmm / kwolekr and myself, find the idea of 'world not big enough' funny and ridiculous, and strongly oppose enlargement. A few players ask for it only because of Minecraft's size.

There's more than just because of Minecraft, such as the fact that exploring horizontal directions is more likely as it's easier to walk kilometers than it is to climb up/dig down. That said,

No, completely different.

Does tell me it's a moot point as apparently I was mistaken about the difficulty of horizontal map... stacking? via realms/subworlds/etc. Actually, I am kind of curious as to why it's harder - It can't be the generation, can it? That would remain a problem even going vertical since any floating islands or cave systems would need to be generated consistently. Probably getting off topic though =/

Icalasari on 2 Oct 2016

I can't help but feel cramped, if not claustrophobic while playing Minetest.

256m vertical height is far more cramped, you can't even fit a mgv7 mountain in that let alone a decent depth of underground plus mountains (mountains would be limited to 128m high, clouds would be at world top, and you could only dig to 128m which is pathetic).

If anything, it seems like if that implementation was included, it would be more work to go out of your way to not include the possibility for mods to also allow it to load along the x and z axis

I'm referring to the difficulty of edge-teleport and seamless terrain edge-matching, as opposed to just stacking differing realms without those 2 features, the 2 are very different.
It's true though that this:

It's possible to stack the same character of landscape vertically to create more land area (without seamless edge-matching ..

.. is easy and similar to stacking differing realms.

paramat on 2 Oct 2016

I think that the increased vertical height minetest has is much more important than the larger horizontal possibilities that minecraft has. However, it still would be a great thing to have.

est31 on 2 Oct 2016

@paramat

256m vertical height is far more cramped, you can't even fit a mgv7 mountain in that let alone a decent depth of underground plus mountains (mountains would be limited to 128m high, clouds would be at world top, and you could only dig to 128m which is pathetic).

I think I'll quote @Icalasari.

There's more than just because of Minecraft, such as the fact that exploring horizontal directions is more likely as it's easier to walk kilometers than it is to climb up/dig down.

The terrain in Minetest is great. I can't help but appreciate how spectacular a 900 meter high mountain looks, but it's not often a great place to build, which is commonly a focus in this game.

Not only that, but a lot of buildings built in Minecraft aren't usually very high skyscrapers. A lot of great buildings can be built in the small vertical height that Minecraft provides. Minetest provides the same space, though, so I guess there's that. (As I mentioned in the post, only large servers and adventurous players really bother going out to the limit.)

But I feel like this is dragging into a discussion about Infinite Worlds, rather than (potentially a lot of, if mods similar to Mystcraft are made) Submaps.

C1ffisme on 3 Oct 2016

Sorry, I didn't read all posts here.
These are my ideas on the topic:
A 'Subworld' has the following properties:

a parent. If it has no parent, some properties (like skybox) need to be assigned with it, else it is a VAE and has translation/rotation values relative to it.
an id (either a string or a number)

When implementing sub-worlds, the database format should be changed too. For a mapBlock there are 4 primary keys (sw_id,x,y,z), therefore the limitation on 31000 would fall( it would break mods relying on pos hashes) This has already been done (#1845).
ATM all coordinates on the LUA side are tables {x,y,z}. We could extend them to {w,x,y,z}. If w is omitted, for node calls w=0/overworld and for players the dimension player is in. This wouldn't break any compatibility.
vector functions should just carry w unchanged. Maybe display warnings when adding vectors with different w.
When VAEs are planned, the mentioned database change is not avoidable, so better make it now than later.
This would also have the advantage that a player that is attached to a VAE has this as w set and all callbacks, raytracing a.s o operate on the context (the VAE) and not on the env.
Even nested VAEs would be possible...

orwell96 on 23 Oct 2016

BTW world wrap (left out, right in) would be a nice thing too

orwell96 on 23 Oct 2016

@orwell96 I don't get the point of parent worlds, in the way that you explain it. Wouldn't it make more sense for VAEs to require a parent and normal dimensions not to? I also don't think VAEs are inherently a part of a dimensions feature, so it should probably be considered separately.

The main reason I support a dimension that is passed around separately from the vector is that it makes the dimension transfer more explicit, since in the implementation I was considering (the client doesn't keep track of different dimensions, and the server just sends them the map of the dimension they are in), the current dimension of something is a completely different beast from ordinary location. A secondary reason that I support separating position and dimension is that the "dimension dimension" of a position vector has no spatial properties, so does not really fit in a position vector. It would be like putting the player's beard length in the position vector. By "no spatial properties", I mean for example that being in dimension 1 brings you no closer to dimension 0 than if you were in dimension 2. And if two vectors have a different dimension dimension, their locations (in x,y,z) are not comparable at all, because they are not even in the same space. There is nothing "positioney" about the dimension a player is in, despite being called a "dimension".

raymoo on 23 Oct 2016

I never called it dimension.
The advantage of my approach is the possibility to represent both VAEs and Sub-worlds with one structure.
Well VAEs maybe don't need a parent but just a w coordinate itself.
I know and I agree that w is not really a dimensional argument that fits into a position table, I just mean that this is the easiest way to implement it.

orwell96 on 24 Oct 2016

Sub-worlds in a separate folder make a lot of sense. In Minecraft, notice how it takes at least 15 seconds to load terrain and such before the player is actually transported. In Minetest, having the game load a separate world without showing the player, whilst showing them either a "Generating Terrain" bar or some portal animation would simulate a "dimension". Maybe call them realms instead?

GreenXenith on 9 Feb 2017

@orwell96 It would be a bad way to implement it because it mixes it up with ordinary vectors that are e.g. accepted and returned by the vector helpers. It would be better for dimension to be passed around separately from position, which would make it easier to achieve some level of backwards-compatibility.

raymoo on 9 Feb 2017

I don't really get this talk about the terminology being used here. However it's implemented, you can call it whatever you like. I'll build a portal to the Nether Potato. :roll_eyes:

C1ffisme on 9 Feb 2017

+1 (I hope this is not a unproductive comment)

tobyplowy on 10 Feb 2017

👎1

I am against the multiple server idea, I understand it would be easy, but easy/hackish implementations aren't always the best.

My reasons:
Multiple servers means multiple servers, thus having to run the same mode once, twice, maybe above 8 at the same time.
Dimensions are better then just stacking so each dimesnion can have a different map gen if needed, and space can be huge, or tiny, without weird boxes up high in the sky
And yes, maybe the idea is like Minecraft, but the way they implement it works, so why do we need to have a different solution because "Minecraft is like that/does it that way"?

MisterXtreme on 21 Feb 2017

👍4 ❤1 🎉1

@XtremeHacker Just an idea: server starts, loads the mods, and then just fork()s... the problem is that we have to support the forkless system too somehow...

numberZero on 22 Feb 2017

@numberZero That could work, but using multiple server running seems like a superglue type "fix", instead of a full on "mechanic repair", get what I mean? :)

MisterXtreme on 23 Feb 2017

@XtremeHacker The problem is that the current API assumes there is only one map. Perhaps the new API should look like map = minetest.get_map("dimension name"); map:get_node(pos)? This way, Voxel Area Entities could be implemented too.

numberZero on 23 Feb 2017

👍1

@numberZero What if we had a 0.5 branch where we could work on this game breaking stuff, and the 0.4 branch for minor stuff, then mod authors could update, and we get some cool features.
TL;DR Have an experimental branch for new backwards compatibility features

MisterXtreme on 23 Feb 2017

👎1

So, I looked on Minecraft, and found the following file structure, I think I can figure out how it works
Root of the world folder, for example "F00Bar"
In "F00Bar" are the following directories (and some random files).
"Playerdata" holds inventory, experience, personal data from an RPG mod, etc
A "Dim0" which would be overworld, and then Dim1 = Nether, Dim2 = End, Dim3 could be a dimension from a mod, etc
So all we need is that file structure idea, and the ability for the Engine to have multiple "sub worlds" loaded from the main world folder, the playerdata folder is a big key to this.

MisterXtreme on 23 Feb 2017

More primary keys in the database / bigger mapblock hashes
Minetest supports databases of different backends, unlike minecra*.
MapBlockHash = WorldID×2^24 + x×2^16 + y×2^8 + z
Would nicely fill a 32bit integer field...
Store assignments to which id is which dimension in map_meta.txt or so.
Why don't we have a 0.5 branch? Some other incompatible changes could be brought through too.
At least we have people working on csm now...
And we need a name for the sub-worlds.

orwell96 on 23 Feb 2017

@orwell96 We have 12-bit mapblock coordinates now, thus 36-bit hash and ±32km world space.
IMO it’s better to use several fields forming a single primary key.
The DB should store most world data: map data, players, subworld ID assignments, and so on. Also note that VAEs should be able to use this technique too.

A "Dim0" which would be overworld, and then Dim1 = Nether, Dim2 = End

MT is not an MC clone. A subgame should define subworlds itself.

numberZero on 24 Feb 2017

👍3

@numberZero I meant as an example of how Minecraft stores that information, subgames/mods would be able to define what each "dimension" actually is of course. :)
TL;DR I meant those dimensions as examples, not saying they should be included by default

MisterXtreme on 24 Feb 2017

👍1

So, any core devs wanna way in on this with the new ideas that have been presented?

MisterXtreme on 25 Feb 2017

👎1

@numberZero Wait, 2^16 is 65536, signed +-32768 (2^15 + sign), so it can't be 12 bit (2^12 is 8192)
But I am also suggesting splitting up the coords in multiple fields (see these issues)

orwell96 on 26 Feb 2017

@orwell96 2^12 = 4096. but that’s mapblock pos limit, multiply by 16=2^4 to get node limit: 2^12 * 2^4 = 2^16 = 65536, thus 64km overall, thus ±32km

numberZero on 26 Feb 2017

~~I don't think it's worth doing when we have lots of unused vertical space where we can store and seal-off another dimension, we need to start using our vertical space.~~

paramat on 26 Feb 2017

@paramat I understand your thinking, I was thinking of the following use cases for dimesnions instead of stacking:
Literal different realms, with different map gen, without being reachable by bean-poling
Stacking means each mod needs to be aware of the other ones and each mod has to be able to "reserve" a spot of vertical space

Also, I see that as a core dev, there is a lot of pressure, what if someone else worked on this, would it be OK then?

MisterXtreme on 26 Feb 2017

I really don't support the idea of stacked realms.

lots of light issues will appear
discontinuity in the maps, fluid issues
player tags visible in completely random places that are irrelevant
requires exotic transport mechanisms

Plus it really just avoids the underlying problem and further reduces the available map space. I'm not saying we shouldn't have floatlands, that's different.

Some sort of multithreaded map manager sounds like a much better approach. I'm still convinced that relaying clients between servers is the simplest method, and with e.g. pqsql backends could result in a performant and easy to maintain setup.

sofar on 26 Feb 2017

👍4

Yeah good points, dimensions are best not stacked.

paramat on 27 Feb 2017

🎉4 👍2

I feel less strongly about needing a shared lua environment between realms, but naively spawning one OS thread per world will not scale if a mod allows the creation of an arbitrary (up to some format or machine constraint) number of realms. Minetest should support the use case where each player happens to be sitting in their own pocket realm (though not permanently, because at that point they might as well play singleplayer).

Also atomic updates on multiple worlds at once will become complicated with multiple lua environments. An example use case would be if you wanted to destroy all shrines of a particular god across all realms. This can be worked around by setting some shared state that says the shrines should be destroyed, but that is an example of the extra complication I am worried about.

raymoo on 27 Feb 2017

👍1

This raises an interesting question: How many “other worlds” can / do we want to support?
This number should be chosen carefully because it will probably not be funny to increase it later.

2^32 will be more than enough for even the craziest use cases I can't even imagine yet. If you visit 1 NEW world every 15 minutes, you will need ca. 400008 hours, or 16667 DAYS to run out of worlds. But it may be overkill if space is an issue
2^16 will be enough for a Mystcraft-like game. That is, part of the gameplay is to travel frequently between many different worlds which get generated on the spot as you visit them, each with different mapgen parameters determined by gameplay actions just as writing linking books, using artifacts, etc. In such a game, the player will easily break into the 100s of different worlds. If you visit 1 new world every 15 minutes, you run out of worlds after ca. 6480 hours, or 270 days.
2^8 is probably still a lot for a Mystcraft-like game, but a long-term player may hit this limit. For subgames with a fixed number of worlds this is probably more than enough forever. For Mystcraft-style, this number is critical: If you visit a new world every 15 minutes, you run out of worlds after 64 hours. Still long, but it is now realistic to hit this number
64 is too low for a Mystcraft-like game but still enough for subgames with a fixed number of “other worlds”
16 should be the absolute minimum, otherwise there is no much benefit for having support for “other worlds”.

Or could this be done in a way that there isn't a hardcoded limit in the first place?

Wuzzy2 on 27 Feb 2017

@Wuzzy2 at your calculations you forgot about servers that have multiple players.

Btw. how will it be saved, in which real something is? Will there be an optional 4th thing in pos table? Or is it too early to think about such things?

Desour on 27 Feb 2017

@Wuzzy2 I think 2^32 is probably overkill, unless you had a server with thousands of players visiting thousands of dimensions. (Which is probably unlikely, I bet more than half of them would leave after trying to punch stone or ask to be an admin before logging off again.)

2^16 is a nice number, but it might still be a little bit overkill.

2^8 would work if you weren't making an exact clone of Mystcraft, maybe if you were making something that achieved something similar for a much more expensive price.

64 means Mystcraft isn't really possible, (I suppose you could do it, if you resorted to saving dimensions in folders within the world-folder that don't get used by the game and then restored them upon entering that dimension. Kind-of a hack, but could work.)

16 is just enough for say, Galacticraft. If you made the planets really interesting and not just a simple chunk of rock with ores.

C1ffisme on 27 Feb 2017

@Wuzzy2 which numbers are you talking about? 😕

RHRhino on 27 Feb 2017

@Wuzzy2 As for me, there are 2 options only:

< 100 worlds. This way, each world may be stored in a separate table.
> 100 worlds. This way world id need to be stored in each mapblock alongside with coordinates, thus plus 2 to 4 bytes per mapblock—not a big difference.

Also in the second approach additional maps may be used for VAEs, as I posted before. Or even something like Chisel and bits dining room, if we ever support scaled VAEs.

numberZero on 27 Feb 2017

@numberZero Whoops, I totally forgot that there are mapblock hashes in the DB and not node position hashes, sorry.
So, xyz block locations occupying 36 bit, which means there are 4 bit left until 40 (the next multiple of 8 and 12 bit until 48 (the next multiple of 16)
-> 12 bit for VAE/Subworld ID = 4096, should be enough.
Having separate tables for sub-worlds would work, but is bad practice.

orwell96 on 27 Feb 2017

@orwell96 oh well, if we keep to storing "hash" instead of coordinates...
Actually, 48-bit numbers present in some RDBMSes, so that may worth considering. Anyway, Lua stores values as doubles so it can't properly store integers above 2^53, that’s near to that limit so 64-bit hashes don’t worth considering.

Having separate tables for sub-worlds would work, but is bad practice.

For small number of worlds that should be acceptable. But still, that’s not the option I prefer, I thought I had mentioned that in the previous message.

P.S. 2^12 = 4096 :P

numberZero on 27 Feb 2017

If you want to add another 8 bits (for a 56-bit hash), 2^20 is about a million.

raymoo on 27 Feb 2017

@raymoo Lua 5.1 can’t handle integers larger that 53 bits as it stores numbers in double-precision floating-point format internally (although LuaJIT seems to support real 64-bit integers, but that’s not the only engine MT can use AFAIK)

numberZero on 27 Feb 2017

@numberZero Are mapblock hashes exposed to Lua?

raymoo on 1 Mar 2017

@raymoo oh well, now my mistake. That were node position hashes. Even worse problem, we use 48 bits now out of 53 available.
http://dev.minetest.net/minetest.hash_node_position

numberZero on 1 Mar 2017

Subworld information doesn't need to be added to position hashes.

EDIT: Because the API doesn't require position hashes to be used anywhere (except for converting hashes back to positions).

raymoo on 1 Mar 2017

@raymoo oh well, then not a problem. so, we can change mapblock hashes to full 64 bits, 16 bits per coordinate — would be enough for a long time. or use per-RDBMS settings, and ints in memory.

numberZero on 2 Mar 2017

Much vertical space is good, 2^16*16 = 1'048'576 nodes vertical space is overkill

How I would separate a 64 bit field based on ideas in here
bildschirmfoto_2017-03-02_22-59-21
16bit dimension = 65'536
12bit mapblock_y = 4'096 = 65'536 nodes (as it is now)
18 bit mapblock_x = 262'144 = 4'194'304 nodes
18 bit mapblock_z = 262'144 = 4'194'304 nodes
=64bit

orwell96 on 2 Mar 2017

Changing the dimensions (not in the sense of subworlds) of the map feels like something even less likely to be implemented than subworlds though.

raymoo on 3 Mar 2017

It also would be nice, if you can have different Subgames per Dimensions.
So you can have for example ctf and Hungry Games on one Server.

Lejo1 on 12 May 2017

👎1

@Lejo1 That would raise a lot of problems, most notably unknown items.

C1ffisme on 12 May 2017

While different subgames would be very problematic, you should be able to have both subgames running on the server (with both ctf and hungry games active) but only apply rulesets to those in a particular dimension (i.e. only handle flag spawning in the ctf dimension)

Anybody feel like updating this from irc knowledge?

thePalindrome on 12 May 2017

@thePalindrome That should be a single “subgame” supporting different rulesets.

numberZero on 12 May 2017

👍2

And would it be possible to connect servers with other servers?
like: minetest.send_player_to_server(player, server, player_hash)
The hash is for creating a player account.
It also should be possible to set that player only can join from a specific server.
Issues about this #5813

Lejo1 on 19 May 2017

👎1

Note: it is possible to add forward compatibility with this feature IMO. What is needed is to move map interface out of the minetest namespace and define some methods to get it.

local map = minetest.get_map("default")
local map2 = some_player.get_current_map()
if map == map2 then
    map:set_node(...)
    map:get_vmanip(...)
    ...
end

The only problem I see currently is that some callbacks might require signature change as well.

numberZero on 23 Nov 2017

@numberZero The most significant challenge with this feature would be not having to eternally rebase because it is going to touch a large part of the codebase. It would need to be done as incrementally as possible, and may need coordination from other devs to require less rebasing. Or just work so fast that nobody has time to cause conflicts.

EDIT: And I feel like the first step would be what you said, except on the C++ side. Changing the map to store multiple maps.

raymoo on 23 Nov 2017

to add forward compatibility

I meant, if the feature is not going in the 0.5.0 release, but maybe in some 0.5.x release.

numberZero on 23 Nov 2017

It won't be in 0.5.0.

paramat on 23 Nov 2017

I don't know why this is still being discussed, it's not needed, will be disruptive, a lot of work, and the implementation seems difficult in either form. Very low priority.
We can already stack realms and separate them with impassable planes, and so then actually use the vertical space we have.
I'm wondering if this request is another MC influence thing, mostly done 'because MC does it' instead of good reasons.
It doesn't seem worth the disruption or effort -1.
EDIT: Sorry for this, temporary bad mood.

paramat on 23 Nov 2017

👎5

I don't know why this is still being discussed

Because it’s wanted.

actually use the vertical space we have.

On the server I play at, many good players merely like to dig down to –31km and to leave a warp receiver there.

another MC influence thing

I don’t play MC. I play MT only, and consider that vertical stacking just awful.

Still, even if multiple normal maps are not considered, VAEs are in the TODO list. They could use the same or similar API. (Unless you want to restrict them to static nodes only).

numberZero on 24 Nov 2017

It may be wanted by a few, that's no reason to add bad ideas (not saying this is bad, just questionable). Any idea will be wanted by someone.
I mean using the space at world top. It may be in use yes, but a server can make the choice.
I support VAEs, better to concentrate on that i think.

paramat on 24 Nov 2017

@paramat with The current Way of adding Dimensions You Can't Get an Infinite amount of Dimensions
I once played this mod for Minecraft a long time ago and it had 50 new demensions it was pretty cool

I don't think you can add 50 dementions with the current system can you?

tobyplowy on 24 Nov 2017

I feel separate worlds/dimensions works well because as the system is now, the more "y-dimensions" you have the less y space you have to play with. Not a problem for most people, but I could imagine it causing issues for larger communities with more mods.

It's a pretty edge case, but from what I understand of the API, that's pretty much the only issue I could see here (that an the fact that "y-dimensions" have a much smaller y than the "overworld" would, which would be a problem more often)

thePalindrome on 24 Nov 2017

👍1

@paramat The current solution is an ugly hack, that only became popular because an alternative didn't exist. By adding a system specifically for dimensions we can add useful API functionality to work with them, and it would be dependable enough to consider in MTG. I don't want to add a moon dimension to MTG with the vertical stacking method.

A lot of people would find this API functionality incredibly time saving and space saving when trying to create "dimensions", "celestial bodies", "sub-worlds", "hub-worlds" or just "private spaces". Plus, the current system cannot stop people from cheating by piling blocks to high dimensions or mining to low dimensions unless you want to do even more hacky stuff.

C1ffisme on 24 Nov 2017

👍3 ❤2

The reason why this is suggested so often is because it's a good thing.

Sub-worlds (I can't stand the term “dimensions”) are a very useful tool to create a completely different experience.
Sub-worlds are useful to create areas of the game which require completely different mapgen rules. Different worlds, really.

Stop thinking Minecraft and start to think of a subgame in which the player can explore many different worlds IN-GAME, all with wildly different mapgens. This would allow for a whole new [i]category[/i] of exploration.
The fractal mapgen (which is actually many mapgens in one) would be great to be explored as a sub-world.
Then there's the possibility of using a core mapgen, but with crazy/unusual mapgen settings.
And of course anything which you could possibly imagine using a LuaVoxelManip.
There are just so many possibilities

The mapgens of Minetest are already very powerful, the problem is just that this feature has been very much unexplored so far.

Realms are never a solution. First problem: All realms are subject to the same core mapgen. This is a VERY big deal.
Basically this means you either can only grab a cave section and just re-used that, or you have to generate EVERYTHING by LuaVoxelManip, which doesn't sound efficient. Except for the realm at the surface (the “Overworld”) no other realm can easily make use of the core mapgen's surface generation.
Second problem is that it's hard to get lighting right with realms.
Overall, realms are nothing but a workaround, not a solutions.

Wuzzy2 on 24 Nov 2017

👍2

My first comment was too negative, edited.

The current solution is an ugly hack, that only became popular because an alternative didn't exist.

Nothing ugly about it, simply a clever use of vertical space. Stacked realms don't pretend to do everything that dimensions could, because they can't. They are not a 'hack' for something else.

I don't want to add a moon dimension to MTG with the vertical stacking method.

That's actually a use case where stacked realms makes sense, because a moon would actually physically be above the normal world, such that you could fly to it in a vehicle. Going to the moon via a portal makes no sense at all.

Plus, the current system cannot stop people from cheating by piling blocks to high dimensions or mining to low dimensions unless you want to do even more hacky stuff.

They can stop that, and it's not a hack, it's just making use of what we have to do something, which is what all mods do.

We could code it such that different y levels have different core mapgens. Light issues can be solved.
Realms are not a workaround for something else, but they can be used as a workaround.

Anyway, i was too negative about this, sorry, i'm fairly neutral.

paramat on 24 Nov 2017

To be fair, there's also downside with sub-worlds: You have to start loading new files again. Realms will be faster to teleport you.

We could code it such that different y levels have different core mapgens.

How? Also different core mapgen params?

Light issues can be solved.

How?

Another problem with realms: All realms are subject to the same daytime. If you want eternal night/day/dusk, affecting the REAL light level, you're screwed.

Wuzzy2 on 24 Nov 2017

That's actually a use case where stacked realms makes sense, because a moon would actually physically be above the normal world, such that you could fly to it in a vehicle.

That doesent really make Any Sense if you would get it like this than you would have to go through all the other layers (realms) and than you would go through the floor of the moon to than finaly land on the moon is it just me or did I miss something :/

We could code it such that different y levels have different core mapgens.

I still racer have actual demention support (becouse all the other possibilities)

Anyway, i was too negative about this, sorry, i'm fairly neutral.

Yes that's correct but don't be to hard on yourself you have to see the possibilities and not the fact that this is in Minecraft becouse creative freedom gos above all :)

tobyplowy on 24 Nov 2017

That's actually a use case where stacked realms makes sense, because a moon would actually physically be above the normal world, such that you could fly to it in a vehicle. Going to the moon via a portal makes no sense at all.

That's why we can have a mod that lets the player build a rocket entity, and the API will teleport the player and the entity to another dimension. Plus, if 1000 cobble is easier to mine and place than your rocket, then what's the point of building that rocket?

Besides that, Minecraft has an almost infinite horizontal expanse of land. Why shouldn't they use that instead of a much cleaner, easier to use API that handles different day/night cycles, mapgen and cheaty dimension hop prevention?

Dimensions can either be professional, clean and easy for new modders to make, or they can require the new modder to work from scratch with incredibly limited potential and a lot of accidental destruction if you do something wrong.

C1ffisme on 24 Nov 2017

👍4

Stacked realms is not as limited as people are suggesting. It could be very similar to "real" dimensions, if the client treated different levels as separate, for example blocking players from going between levels without teleportation. The main reason I prefer "real" dimensions is for use cases like MC's Mystcraft, where players can create new dimensions as a gameplay mechanic and shouldn't be limited to the amount of vertical space in the map.

raymoo on 24 Nov 2017

👎1 👍1

If the coordinates were boosted to something like 32 bits, though, stacked realms may be able to fill even the Mystcraft use case by sheer vertical volume. Managing stacked realms, with different mapgen, skies, etc. may or may not be simpler than just making them separate maps.

raymoo on 24 Nov 2017

I support VAEs, better to concentrate on that i think.

@paramat If you don’t mean to solidify VAEs on creation (essentially making them a sort of oversized LuaEntities), they need their own maps. Well, unless you want them to be stored in the main map somewhere at +31km. But that way you will have great trouble with entities whose content needs to cross mapblock boundary (that’s possible even with small entity if you modify it).

numberZero on 24 Nov 2017

Yes i can see the close relation, i don't really stand by that statement anymore.

paramat on 25 Nov 2017

Different dimensions are customisable as hell.
Another thing: there is mc beta 1.7.3 server (year 2011 distrib, plugins, retro), it has portal to their previous map on spawn where you can actually play, portal to skylands (cough floatlands), nether etc, you can hook anything basically and then google "RF Tools dimensions" - will blow your mind, you can construct your own dimensions in game and visit/live in them.

Fixer-007 on 25 Nov 2017

👍1

I agree with sofar on this. The fourth co-ordinate approach seems to be a huge, disruptive and complex task, and somewhat unnecessary, the feature is not important enough to be worth the complex approach.
Sorry to be realistic and practical about this, we have a long term severe lack of dev time and can barely cope with basic maintenance, and already have the CSM project to cope with.

Major projects like VAEs or enlarging the world size are much more deserving of attention, if or when those are practical and we have enough dev time to cope with those.
I'm fine with dimensions if it can be reasonably simple with no compatibility breakage or world format changes.

paramat on 6 Mar 2018

The fourth co-ordinate approach seems to be a huge, disruptive and complex task, and somewhat unnecessary, the feature is not important enough to be worth the complex approach.

Okay, for the record, when I wrote this post almost 2 years ago I should have realized the fourth-coordinate thing was confusing terminology that didn't mean what I wanted at all... And I blame myself for that. I actually just wanted entities and players to have a variable that would determine which dimension they were in... :P

Still, I get the point that this would be difficult.

C1ffisme on 6 Mar 2018

@paramat Another issue with stacking realms is that higher realms can cast shadows. In practice it may not be noticeable but in theory it could still happen. I would like to be able to create regions whose contents are transparent except when the camera is inside them.

adlocode on 27 Mar 2018

👍1

I think there's a way around that, although i'm not saying stacking is a substitute for dimensions.
In the LuaVoxelManipulator there is a bool for 'propagate shadow' which allows mapgen-created shadows to be stopped at a chosen level.
To stop build-created shadows propagating you would perhaps need an inaccessible buffer zone between realms a few mapchunks thick, probably made with airlike undiggable 'walkable' nodes.

paramat on 27 Mar 2018

Another issue with stacking realms is that higher realms can cast shadows.

If you have, say, ten dimensions vertically, you will need to go up thousands of blocks to cast one tiny shadow, it is not practical for griefers IMO.

Fixer-007 on 27 Mar 2018

It's possible to implement dimensions in a way that doesn't break mods by mapping dimension coordinates into 3d space. So dimension w=2 could have (0,0,0) at (0,100000,0). This is better than a stacked realm in that it would not be possible to travel to it normally and there would not be shadow problems

rubenwardy on 27 Mar 2018

@paramat What property do the nodes need to have so that shadows do not propagate? Will this prevent the nodes above them from casting shadows? Of course, it must not prevent all shadows within a given realm; it must only prevent shadows being cast on lower realms.

adlocode on 27 Mar 2018

@rubenwardy the problem with that approach is that every API function that includes pos needs to be modified, or the whole definition of pos itself needs to include realm as an extra table member. Mapgen and a whole bunch of core engine parts also likely need massive overhaul.

The shadow part of this problem is really minor compared to e.g. the need for different noises in the secondary realm, and/or a completely different mapgen altogether.

sofar on 28 Mar 2018

You actually only need to modify the few functions that convert to and from positions to change how positions are converted from and to the engine.

But that's not what I meant, I was intending the engine to also use this mapped position system in most places. It's the position hashing, mapgen, and float sizes that need to be changed

rubenwardy on 28 Mar 2018

adlocode, the 'buffer layer' between realms is only to stop players entering it and being able to propagate shadows from realm to realm, also it stops players being able to get close enough to see stuff in other realms.
Nodes themselves cannot stop shadows propagating.

The stopping of shadows during lua mapgen is done in the LuaVoxelManipulator
https://github.com/minetest/minetest/blob/040b878cd5f77f6b44622b7c00255b4981500dc8/doc/lua_api.txt#L4608
Or in a core mapgen like mgv7 is done here
https://github.com/minetest/minetest/blob/040b878cd5f77f6b44622b7c00255b4981500dc8/src/mapgen/mapgen_v7.cpp#L356
This is to stop floatland shadows affecting the surface world.

In both cases the propagation is only stopped at a certain y values (between realms).

paramat on 28 Mar 2018

@paramat So, build generated shadows are stopped by being so far away that the realm is not rendered, i.e. it's more a fluke of rendering? It would be nice to have a more "solid" guarantee than that.

adlocode on 28 Mar 2018

Not really, when a node is placed, shadows are added to a certain distance below the node, actually not sure how far, maybe a mapchunk? If the buffer zone is larger than this distance then build shadows cannot pass the buffer zone.
I want to test this now to be sure :]

paramat on 28 Mar 2018

My main problem with stacking is that with 65k nodes vertically, there isn't a lot of room for dimensions.

raymoo on 30 Mar 2018

👍1

Yeah, because 64 1k high dimensions, or 6.4 10k high dimensions definitely isn't much room.

benrob0329 on 30 Mar 2018

My main problem with stacking is that with 65k nodes vertically, there isn't a lot of room for dimensions.

If we take current minecraft with 256 blocks of height as bare minimum for playable game (and lets be honest, it is totally playable), that will give you maximum of around 250 dimensions, that is more then enough, but if you want much thicker world, like 2000 nodes thick, you can fit 30 dimensions easily.

But if devs somehow allow managing separate database for each world, that will be fine too (as well as allowing bigger worlds).

Fixer-007 on 30 Mar 2018

250 is not more than enough for a mod like Mystcraft, where each player may be creating more dimensions.

raymoo on 30 Mar 2018

👍4

My opinions on a few technical points while reading through this:

Naming: most naming conventions are fine by me. The ones I don't like are dimensions (easily confused with spatial dimensions, which will make searching online and on a document a major pain) and sub-worlds (because then we're going to be referencing the main world as a "sub-world" which makes no sense at all... it's a sub-world of what if it's the main one..?). Planes and worlds are fine by me, for example.
API: keep the current API with XYZ coordinates - It just feels right and keeps compatibility. Add minestest.worlds with minestest.worlds['default:default'] being the main world (or similar names). Add a player:get_world() too and you can then easily work with any world at all by using XYZ coordinates.
Stacked: mods don't need our help to create stacked worlds - they already do that just fine. Stacked worlds are great if mods/games want to create caves/hell/heaven/whatever but that can be done with the current API as-is. This issue should be about fully independent worlds that aren't accessible physically.
Server-based: I am very strongly against this. There have been plenty of arguments made already but I haven't seen anyone talk about how much of a huge pain in the ass this will be to configure. If I want to use a Mystcraft-like mod, do I need to setup, host and pay for a billion servers? What if I'm using a mod that depends on a mod that allows for planar travel and I don't even know it?! What happens when my players try to jump to another world with no server?! One server multiple worlds is the only way to go about this.
Priority: I love this feature and it should definitely be supported at some point. However, I don't think this needs to be even a 1.0 milestone. It's going to be a lot of work and should not block 1.0. Make it the major 2.0 milestone maybe, even if it takes Minetest a decade to eventually get there.

tukkek on 24 Jun 2018

@tukkek Generally, I agree with your points.

On API, I don’t fully understand what kind of object minestest.worlds[...] you mention should be. I personally imagine it as a map, with get_node and other such methods, but nothing like node/item/whatever definition management stuff.

On “server-based,” there is technical aspect and logical aspect.

Logically, these worlds must share a lot of important data like node definitions. Only map-specific things like ABMs may differ, but should be shared by default.

Technically, spreading worlds over several servers gives the possibility of running them on separate machines. But that may only be wanted on really heavily loaded servers; we are far from being ready for those. Currently, several threads in single server process should be enough to handle many worlds, as long as only few of them are under load. And, well, multi-threading Lua is challenging.

P.S.

What happens when my players try to jump to another world with no server?!

fork(), I suppose :P

numberZero on 24 Jun 2018

Note SemVer MT1.0.0 was actually around 7 years ago :) we're not working towards 'finishing', 'releasing' or 'completing' MT. We have just changed version to 5.0.0 to reflect this.

paramat on 24 Jun 2018

what kind of object minestest.worlds[...] you mention should be

I was thinking a basic Lua table (Table from ltable.c?) holding World objects (or however they end up being named). Now that I think of it though, it might be better (or just as good) to have a few functions like minetest.get_world(name), minetest.add_world(name), instead. A basic table is simpler to code/use but also easier to mess up from inside a mod by directly writing to it, if it's exposed like that. I guess either approach is fine?

We have just changed version to 5.0.0 to reflect this.

Thanks for the update, I had no idea :D Agreed that 5.0 looks like a full-fledged game, especially with the in-game content browser!

tukkek on 25 Jun 2018

A basic table is simpler to code/use but also easier to mess up from inside a mod by directly writing to it, if it's exposed like that.

@tukkek There are index and newindex metatable entries usable for protection. Take a look at how items and nodes are registered (builtin/game/register.lua IIRC).

holding World objects

My question was about that World object itself, what should it have and do?

numberZero on 25 Jun 2018

I expect a World object would have a lot of the minetest API objects moved onto it, like anything involving a XYZ position, nodes, entities, etc. The default API would just delegate all those calls to the main World, if we want to keep backward compatibility. I can't imagine any other way to implement many Worlds.

This could cause trouble for mods that expect a player to be in the main world so they can spawn something near the player, for example, since if they don't update, they will spawn things in the main world and not wherever the player is. However, there really is no way around it, because if the methods are going to default to anything, they can only default to the main world anyways.

The alternative would be to remove all current API methods and move them into World objects but this would break nearly every single mod out there. This requires older mods to be upgraded if they want to support many-worlds but I think it's OK.

tukkek on 25 Jun 2018

The existing functions could just call things on the "main" world.

Multiworld servers could take a long time to finish, but it's probably better that it's started early, since it will probably touch a large part of the codebase.

raymoo on 25 Jun 2018

I thought i had this idea first, then i searched through issues and noticed: someone else had that idea 2 years ago... :D

niansa on 30 Jun 2018

😄1 👎1

I change my mind about defaulting to the main world, the global API functions that would be moved into world object methods should be removed. Allowing dimensions will break mod functionality either way, as @tukkek mentioned, so it's better to explicitly break the API because it will be easier for modders to find the parts of their code they need to change. Breaking the API will change logic errors ("This dirt node spawned in the wrong dimensions") into crashes with line numbers and causes.

raymoo on 24 Sep 2018

The only 'dimensions' i would support in the short/medium term is dividing up our vertical space and stacking, which could be done now with a new layered core mapgen that creates impassable barriers of "ignore" between that also stop shadows propagating.

I don't support any of the complex methods discussed here.

Ironically, MC only has dimensions because it doesn't have the vertical height that MT does. The nether is obviously a 'hell' so would have been placed deep down. Aether was meant to be above. The End could have been stacked above beyond a barrier.
Other modded-MC dimensions could also have been stacked behind barriers.

So the demand is created by another game that only has completely separated dimensions due to lacking a MT feature, think about that, it's highly ironic.

If we eventiually move to s32 co-ords, which is actually somewhat likely, we would then have more vertical space than you could ever need for stacking dimensions. So that would seem the best route to take.

Please don't think about MC too much. It generates too much desire for features in MT and we are not a professional development company valued at 2 billion dollars, yet many users expect the same level of features, quality and development. MT users need to restrain their expectations, this is a simple voxel game engine.

paramat on 24 Sep 2018

👎3

I want to bring up that some mods from Minecraft allow an arbitrary number of dimensions to be added during gameplay (actually I already brought this up earlier in this thread), which may be too many for the current amount possible (256 dimensions of height 256, and horizontally covering the entire space). I do agree that dimensions like this are a low priority item. I don't see many other major features (or features in general) that are currently listed as high priority though, only CSM.

raymoo on 24 Sep 2018

While those "dimensions" are not a required feature in first look, they can certainly be useful in some situations where there is no clear height order of dimensions:

On a CTF server, multiple games are to be run simultaneously, but checking the coordinate range each time is a hassle. Aside from this, maps could easily and efficiently be reset if there was a "copy dimension" function (keep untouched map in one dimension and copy it on new game)
In a Doctor Who mod, it should be possible to create multiple Tardises, which are "bigger on the inside" and thus require a dedicated chunk of map. Settling those in squares at -31000 would be possible, but painful.
I think creating a layered mapgen is more effort than adding dimensions
In some MC server, there was one "farm world" dimension that was reset each night and one dimension using Citybuild as "build world". Yes, we could layer this, but we would need to heavily adapt and reprogram plot_system for that.
With the layering scheme, conflicts are pre-programmed as long as there doesn`t exist a central API to use
Dimensions could be a step forward to VAEs, so that a VAE is represented as dimension in memory and "included" into another dimension.
Mentioned earlier: What about different TimeOfDay in particular dimensions?

I am not opposed to the scheme of stacking Hell, Overworld and Heaven in one single "main" dimension, but I think the effort is worth it in long term. However, I agree that this is not a priority item.
All occurences of "dimension" should be swapped out by "subworld"

orwell96 on 25 Sep 2018

👍1

I think the idea with stacked worlds is that Minetest itself would provide support for stacked mapgen, something it would still have to do if separate-map worlds were implemented.

raymoo on 25 Sep 2018

👎1

I want to bring up that some mods from Minecraft allow an arbitrary number of dimensions to be added during gameplay (actually I already brought this up earlier in this thread), which may be too many for the current amount possible (256 dimensions of height 256, and horizontally covering the entire space).

What MC can do is irrelevant, MT doesn't have to match the dimension-number capability of MC. This is a ridiculous argument.

I don't see many other major features (or features in general) that are currently listed as high priority though, only CSM.

It's not a case of other major listed features, the fact is we are struggling with absolute basic maintenance and getting a release out. So we have to prioritise even more.

In a Doctor Who mod, it should be possible to create multiple Tardises, which are "bigger on the inside" and thus require a dedicated chunk of map. Settling those in squares at -31000 would be possible, but painful.

Not at all painful. Small volumes are perfect for placing at a world's extremes, behind a barrier.

I think creating a layered mapgen is more effort than adding dimensions

It isn't, i've almost written one already, it's quite easy for me.

~I think the idea with stacked worlds is that Minetest itself would provide support for stacked mapgen, something it would still have to do if separate-map worlds were implemented.
No, layered mapgen would not be needed with separate map worlds as it's the alternative solution.~

paramat on 25 Sep 2018

I was responding to @orwell96's criticism of stacked worlds

raymoo on 25 Sep 2018

The nether is obviously a 'hell' so would have been placed deep down. Aether was meant to be above.

Right. But:

The End could have been stacked above beyond a barrier.

That would be a work-around.

So @paramat, please don’t mix “we don’t need that” with “we can’t do that.” Your arguments I see are mostly of the second kind, and that’s totally valid. But arguments like “we don’t need yet another MC feature” are certainly not valid per se, but need a very good explanation.

Anyway, this seems too complex for any near future, so the issue should probably be postponed in some way. GH doesn’t seem to support that directly, though; maybe close this issue but add a special label meaning “reopen if have a solution?”

numberZero on 26 Sep 2018

What does "have a solution" mean? I think there are known feasible ways to implement large numbers of dimensions (e.g. 32 bit height or have separate maps), just they would cause some disruption and mod API breakage.

raymoo on 26 Sep 2018

I meant, have a working solution. That is, not just words, more like PoC code.

numberZero on 26 Sep 2018

I feel like people won't want to work on an implementation of this currently, since it sounds like it would be rejected for breaking too many things. If the issue is then closed then the discussion would never get to the point where it's clearer what would happen to a PR.

raymoo on 26 Sep 2018

raymoo sorry for my misunderstanding.

Discussion can continue here even if this is closed, closing just means it bothers the core devs less.

paramat on 27 Sep 2018

I am no longer wanting this feature. For now.

Wuzzy2 on 27 Sep 2018

It's possible to implement this without breaking mods by using virtual positions. Instead of adding a 4th co-ordinate, use different areas of the Y value. In effect:

lua_pos = {
    x = engine_pos.x,
    y = engine_pos.y + engine_pos.w * 64000,
    z = engine_pos.z,
}

This is similar to the slicing solution, but at least here the engine knows about separate dimensions and can stop light spreading and such

rubenwardy on 27 Sep 2018

👎2

I feel like there's a danger there of mods doing things across the (virtual) world boundaries, like connecting cables across worlds. Also it seems to me that it's basically equivalent to increasing the y width and stacking worlds, only difference (in relational db) being that you have a separate column for dimension instead of it being part of the block position. Which means stacked worlds would have the same boundary problems, so maybe that's a point against that too.

raymoo on 27 Sep 2018

👍1

@rubenwardy that’s still slicing. My first though was that now slice size is some opaque value, so that it could be increased arbitrarily, but as mods may want to store positions, that’s not the case.

numberZero on 28 Sep 2018

Closing for various reasons but you are all very welcome to continue discussion.

paramat on 8 Oct 2018

😕1

How are we supposed to "continue discussion" if you've closed an issue with 29 participants and dozens of comments without as much as mentioning a single one of your "many reasons" for doing so?! How are the inevitable new issues requesting this feature going to be addressed? Is this never going to be implemented in Minetest and if so why?!

tukkek on 8 Oct 2018

Discussion can continue here even if this is closed, closing just means it bothers the core devs less.

Wouldn't it bother core devs the same amount if discussion doesn't stop?

raymoo on 8 Oct 2018

Closed partly due to no stated core dev support for 2 years, but i'll reopen and ask if there is any core dev support.

It's probably more of a postponement as suggested here https://github.com/minetest/minetest/issues/4428#issuecomment-424825793 not a refusal. Closing means it doesn't bug the core devs as an issue needing attention, and doesn't add to the overwhelming issue number, but discussion should continue of course.
This is non-trivial, a big task, no clear implementation, low priority, at a time of very slow dev, so highly unlikely to be worked on for quite a while, seems something for the far future, so seems best 'closed but not disapproved'.

paramat on 8 Oct 2018

Seems to me the simplest solution that breaks the fewest mods would be @paramat's suggestion of s32 world coordinates, stacking the realms vertically, and blocking user movements across the borders except by teleporting.

Horizontally, s32 would give you a huge, 4.3M km x 4.3M km horizontal space. I would suggest that the vertical axis be partitioned into 256 km tall slices (instead of the previously-suggested 1 or 2 km, or the current 64 km). Note that I said 256 km here, not 256 m. This would give you room for 8200 slots in which to put your realms, each being tremendously larger than our current limits. Sure, you could further subdivide those slots for realms that just don't need to be so thick, but even 8200 should be enough for even extreme use cases.

The "base" realm, whatever you want to call it, would still have its sea level at the usual Y position. That way, from a mod standpoint, nothing would change in single-realm worlds, and very little change would be needed for multi-realm support, because the mod still only has to deal with 3-dimensional coordinates, and having a 256 km thick slice for a realm means that no mod in existence will have had the option to poke its nose into what could now become some other realm (barring coordinate bounds checking bugs, of course).

In fact, the only thing I can think of that could screw up, from a modding API standpoint, are the hash position functions, given the previously-mentioned 64-bit Lua numeric limit. I think the solution here is to outright, deliberately "break" those calls, making them accept and return position hashes in a form Lua can more easily deal with, instead of integers.

I suggest hexadecimal strings for the hashes, padded to 64 bits per coordinate (just in case :wink:). Since we have negative coordinates, I could suggest two's complement notation, but Lua handles that weirdly, so let's just set the high bit of the padded number if it's negative. Prefix each coordinate with its axis name and jam it all together. I.e. hashing a position of say (12345, -67890, 98765) would return the string, X00003039Y80010932Z000181CD.

Trivial to encode/decode even without the API calls, effectively unlimited range and future-proof, and perfect as a Lua table key.

Seems like the perfect solution to me.

I can't think of any mods that would break just because of larger coordinate values. Mods that use the position hash functions of course would need to be fixed, though but those are a tiny minority.

Sure, some mods would need to be made realm-aware, particularly those the do node placements based on what's in the vicinity of the placed node (this of course applies to VM operations, also), but then that would only be to make sure they place the right things. Where something gets placed should be correct because, again, we're still talking about 3-dimension coordinates. It just becomes a trivial matter of what to place, just so that the thing being placed fits into the theme of that realm. You don't want your mod to, for example, grow a "tropical paradise" realm's palm tree on the shore of some frozen-over lake in an "ice age" realm, or cause a node to burst into flames in the vacuum of some airless moon realm.

VanessaE on 9 Oct 2018

👍3 ❤1

Here are some other classes of mods that would need to change:

Tech mods with pipes / wires - need to check that they aren't connecting across realms. Might be hacked around by putting a buffer layer of blocks between dimensions
Mods that check distance from things that could be far away - they would need to filter out things in other realms. Example: City mod that prevents cities from being built within 1000 nodes of each other
Mob mods - don't have mobs follow players in a different realm
Other stuff not distinct enough to mention

I'm not sure about the 8200 number being enough for "extreme" use-cases. If you had 100 hard-core "Mystcraft equivalent" (whatever it would be in MT) players, and if they hopped between 5 new myst worlds every play session (to find one with properties they like), it would be filled up after they had each played 17 sessions (I think). However, given the size of the Minetest community I don't think this extreme case would happen. With one player they would probably get bored before filling up all the slots.

raymoo on 9 Oct 2018

Tech mods with pipes / wires - need to check that they aren't connecting across realms. Might be hacked around by putting a buffer layer of blocks between dimensions

Do you mean 256 km long pipes?

Mods that check distance from things that could be far away - they would need to filter out things in other realms. Example: City mod that prevents cities from being built within 1000 nodes of each other

Maybe, esp. if they compute 2D distance only (3D distance will be huge).

The good thing is that that may be implemented easily in the core, and then we can provide a convenient API emulating true realms to facilitate transition. That’s probably easier than doing the opposite, i.e. making realms and a compatibility API emulating slices.

numberZero on 9 Oct 2018

Regarding distance/coordinate checks, that's why I suggested a 256 km thick realm - no mod currently in existence will deliberately access coordinates beyond that, because... well.. those coordinates don't even exist yet, so all that would remain would be bugs of one sort or another.

Of course, that doesn't address running a tube/wire/etc just a few meters to cross a boundary, but that is easy to protect against:

Make the realms 256 km thick, but don't generate the bottom-most one-mapblock- or one-chunk-thick portion of the realm. That way you create a dead zone between realms that just looks like air (one should reasonably assume there will be some kind of "bedrock" above the dead zone, underlying the realm). Bingo, instant border crossing protection, and without involving walls or armed guards. :wink:

VanessaE on 10 Oct 2018

What is if a player teleports to another realm and looks to the one he left and activates full view range? Looking into another dimensions wouldn't be nice. The same is with player names.
Problems with mods could appear if a mod adds something per 2d coordinate (like travelnet elevator).

Desour on 10 Oct 2018

That seems trivial to fix, just block the rendering engine from showing anything outside of the current realm. Just because everything's mapped to normal 3d space doesn't meant the client can't be made aware of what realm the player is in. In fact, you could simulate this by having the client just discard all cached map data when the player moves between realms.

VanessaE on 10 Oct 2018

I think there should be a function that returns some realm number for a position. Then mods like the travelnet elevator would be able check that the other elevators are in the same realm (though the mod would still have to change)

raymoo on 10 Oct 2018

👍2

From what crossfire does changing realm is not a major problem. Either the client implements a realm aware cache or when you change realm it drops the cache.

Will end up calling realm a "pocket world"/pocket a bit bare with me.

Mod issues there is a fairly straight forwards set of work around.
1) Each realm be like it own server.
2) Player data stores mod added data are realm tagged.

Why would I want each realm to be like it own server. This is so a realm can in fact run it own unique mod settings. This also results in no mod modifications required.

Architecture will be fairly straight forwards when you think everything is a server.
1) Something like load balancer for http that users log into. This forwards connection with authentication and possible player storage to server user will be on.
2) server processes that pocket/realm can send message back to load balancer to say connect this connection to this pocket/realm space.
3) the load balancer is responsible to spin up pockets/realms that are currently inactive.
4) server processes if there is no require on going actions to shut selves down.
5) client has to understand drop cache or realm particular cache. Make client be able to display current realm so players know where they are.
6) shared messaging system has to exist between the server processes so players in a collection of realms can talk to each other.
7) some API to request a pocket/realm/server change.
8) some new block/world data added to mark allowed enter points.

This could also be a true load balancer where everyone is connecting to 1 IP address and its being redirected to many internal servers. Yes this could allow the main world to be broken into segments.

Biggest work here will be the shared messaging and shared player storage. Yes crossing pockets/realms does not mean your inventory has to follow you in all cases but when it meant to it better..

Its a lot harder to make 1 server have multi pockets/realms inside. As the request states its already heavy at times supporting a single world on a single server. Lets create pockets/realms with the design of making spreading load out to multi servers simpler and transparent to players.

I cannot really see why mods would need to perform actions across the pocket/realm boarder out side using player inventory/data. So simple set to realm/pocket at X entry point should be all the API required with possibility of being refused.

Lets say we have a pocket world that is part of the player data. This we in crossfire. This allows a play to have a private pocket world that they can enter. This can be a bank for storing the items they have collected that cannot be accessed by other players.

Player in mine-test can be allowed to claim X area of map for protected constructed building. Lets say we let players built the building they want in a pocket world full creative tooling while in the personal pocket world. Now they have to create a alter and offer up the materials to bring what is in there personal pocket into the main map. Of course normal world they don't have creative mod.

Of course while a person is in their personal pocket they still would want to be in the chat on the server.

There could be other uses for a pocket like this for like lets say your server has a town with home blocks the players who are logged into the server have there houses mapped into the town. When they log out their house magically disappears with them and the new player to login house takes it place..

Also shared player pocket create could be useful for player made dungeons/quest spaces. So lets say you have design a quest area for 6 players having even just 7 players enter can make the quest way simpler than it should have been. Now if you have 12 players wanting to use that area at once being able to spawn two copies of the pocket allows all 12 of the to play that area at the correct difficulty.

https://en.wikipedia.org/wiki/Restaurant_to_Another_World
Also comes to mind. The door here returns the characters who enter it back to exactly where they came from yet it exactly the same space. So with pockets could create like a shop with multi doors all over the map.

If each realm/pocket is own server process this would mean they could be reset without resetting everything.

Lets say you wanted to have a portal mod game on a minetest server currently you have to load that server wide. Realm/pockets would allow games inside the server the person logged into to use mods just for those games and if those mods play up only effect the players in those pockets.

This feature could make minetest a lot more fun and unified.

oiaohm on 18 Dec 2018

I cannot really see why mods would need to perform actions across the pocket/realm boarder out side using player inventory/data.

Even with only shared player data, there are significant problems that mod writers, not just engine developers, would have to deal with. Here's an example:

A messaging mod stores a player's inbox in their player attribute (not sure whether you can even get the player storage of an offline player but pretend you can)
To store a message, it retrieves the player attribute, which is a list. It then adds an element to this list and puts it back in storage.

Now, consider could happen when players A, B on two different servers want to send a message to the same player C:

The server with player A fetches the list of messages
The server with player B fetches the list of messages, which does not have A's message because the other server hasn't updated the attribute yet
The server with player A stores the new list containing the message from A
The server with player B stores a new list containing the message from B. It doesn't contain the message from A because when the server fetched the list, it didn't have the message from A.
The player is now missing the message from player A.

Basically it forces mod writers to deal with concurrency issues. Something like SQL transactions could be implemented, but it would put a large burden on modders because they would need to use them for almost all code involving player attributes or inventory, if they wanted to be safe with multiple servers. There would be lots of forum posts complaining about strange, hard-to-debug corruption of their player data, and then somebody would suggest after a page of forum posts that it might be due to unsynchronized player data accesses. After another page the original asker would give up trying to figure out transactions (or maybe successfully rewrite their mod to have correct synchronization).

Player inventory and attributes are not the only things you might want to affect across servers, though. Here are some possible examples:

Clan mod that shows online clan members using HUD elements
Voodoo mod that lets you damage players by messing with a voodoo doll
Summoning mod that lets you summon a player to you after you make a contract with them
Player radar that tells you where a player currently is

I don't think these introduce any new problems for modders though, if there was some messaging system for use by mods (not player chat), and some kind of shared mod state. It would just have the concurrency and synchronization problems that also apply to player attributes and inventory.

I've described these problems somewhere else but not in this thread apparently.

raymoo on 21 Dec 2018

@oiaohm Can you explain what does Crossfire do? (And, BTW, which Crossfire do you mean?)

On technical part, it’s more problematic than you show, but possible. Things to consider are:

What to do with player’s inventory?
How to perform cross-server operations, like those raymoo mentioned?
How will these worlds interact, if the mod sets are different, or settings are different?

There could be other uses for a pocket like this for like lets say your server has a town with home blocks the players who are logged into the server have there houses mapped into the town. When they log out their house magically disappears with them and the new player to login house takes it place..

How would you implement that with the multi-server architecture?

Yes this could allow the main world to be broken into segments.

Totally unrelated. Actually that’s the opposite of what you suggest: several ~realms~ segments in a single world with unified mod set and environment.

These problems are technical. But even worse, gameplay won’t get a lot from such multi-realmity; if these don’t interact, you could just share the chat (e.g. using some IRC mod and a single channel for several servers). Transferring a player is easy as well: just let him log out of one server and log into another one, as a single operation. Authentication DB may be shared to prevent various collisions, that may even be possible already.

numberZero on 21 Dec 2018

Do any core devs support this? If so please add the label.

paramat on 24 May 2020

The way to implement this and keep backwards compatibility would be to use a technique similar to virtual memory mapping.

The engine would allocate a virtual base position for a dimension, for example, { x = 45000, y = 0, z = 0 }. Mods can continue to use old position functions without needing to know about a new co-ordinate dimension, and will be able to index all positions.

The C++ code would translate from global positions (ie: { x = 45000, y = 0, z = 0 }) to dimension-local positions (ie: { x = 0, y = 0, z = 0, d = 123 }) when minetest.get_node is called, allowing the Lua API to keep compatibility and the C++ to work as needed.

New mods can use new APIs to use dimension-local positions:

on_punch = function(pos)
    -- Most mods will be able to use this
    print(dump(minetest.get_node(pos)))

    -- Otherwise, you can convert from a global position to a dimension-local position  
    local dim, dpos = minetest.get_dimension_by_pos(pos)
    print(dump(dim:get_node(dpos)))
end,

Alternatives:

Versioned APIs - A mod would declare an API version in mod.conf, and the mods function environment would be set to it. You'd need to use dirty metatable hacks to ensure that APIs like minetest.is_protected remain consistent.
An environment for each dimension, with all mods initialised in it.

rubenwardy on 24 Jul 2020

👍1

No core dev has clearly supported this feature, just discussed how it could be done, or what implementation they prefer if it was done. But one of sofar's comments https://github.com/minetest/minetest/issues/4428#issuecomment-246433535 suggests he might support this, and i suspect this is supported by a core dev, so i have changed the labels.

paramat on 24 Jul 2020

@oiaohm Can you explain what does Crossfire do? (And, BTW, _which_ Crossfire do you mean?)
http://crossfire.real-time.com/
This is the crossfire I was referring to as in Crossfire RPG. It uses a lot of pocket space.

On technical part, it’s more problematic than you show, but possible. Things to consider are:
* What to do with player’s inventory?

There is not exactly straight answer to this. You access a pocket in some cases you inventory that you just had is simple gone and now the inventory of the server the pocket lead you to is there.

Answer to inventory is the server/pocket you jumping to classed as allowed to share the same inventory or not.

* How to perform cross-server operations, like those **raymoo** mentioned?

When it comes to cross server messaging you have to allow for a bit of lousy. There are going to be times when communication messages do get lost when you are in different pockets/servers. This can be reason while you will try to be in the same servers. Same issues happen with IRC splits.

Clan mod that shows online clan members using HUD elements
This one would most likely need a fix. Because you would need to know that Clan members are on-line just in a different server/pocket.
Voodoo mod that lets you damage players by messing with a voodoo doll
Summoning mod that lets you summon a player to you after you make a contract with them
Player radar that tells you where a player currently is
These are simple fix. If both parties are in the same pocket/server they work. If no its the player who on a different pocket/server to the mod acts like they are not connected at all. Radar might show a last connected location as in where player jumped out into pocket space and will return in future if updated for that.

Performing voodoo/summoning across servers get painful to implement and could be path to exploits.

* How will these worlds interact, if the mod sets are different, or settings are different?

Nothing forbids using a pocket door to change world resulting in your prior inventory being left 100 percent behind.

So I would have inventory only stay if its compatible and it makes sense.

There could be other uses for a pocket like this for like lets say your server has a town with home blocks the players who are logged into the server have there houses mapped into the town. When they log out their house magically disappears with them and the new player to login house takes it place..

How would you implement that with the multi-server architecture?
Comes down to how you implement it.

If you look at crossfire you have another choice. 1 house on the map is everyone house. Inserting a 3d space in and out of a map could get too messy if that is too messy. Now have a door that basically lets a player step into a small world that is hidden in his/her inventory and out of the main world. Of could allow other players to be invited in just other players cannot add or remove anything.

Remember I am allowing that when you jump off to a different pocket/server that you have now other than the fact you are still on chat have basically disappeared to everyone who was on the server you were on. Of course global network wide chat does not have to be the same a server/pocket wide chat.

Yes this could allow the main world to be broken into segments.

Totally unrelated. Actually that’s the opposite of what you suggest: several ~realms~ segments in a single world with unified mod set and environment.

I am not looking at 100 percent unified mod set and environment.

These problems are technical. But even worse, gameplay won’t get a lot from _such_ multi-realmity; if these don’t interact, you could just share the chat (e.g. using some IRC mod and a single channel for several servers). Transferring a player is easy as well: just let him log out of one server and log into another one, as a single operation. Authentication DB may be shared to prevent various collisions, that may even be possible already.

The more interact between servers the harder it comes to implement. What you listed here would be where you would start doing a pocket setup. Particularly allowing a door that allows a player to walk out of one server and into another without having to do anything else. Think about a lobby world where you have your player skin and basically none of your inventory and you are able to see a setup world with the types of games you can go play setup doors to walk though. A space like this may not be that big so you might want to limit each running lobby to like 20 players at a time so its practical to move around.

The inventory bit shared between servers has to be done very carefully to avoid item duplication problems.

First stage is getting that you can in fact walk between servers and talk in a standard way being a group of servers. Yes the means to set door in one server and a door in another server and link them so that when you walk into a door on one server and it transfers you to the other server you come out the door/location at the other end.

Trying to make inventory sharing and all existing mods just work multi server straight away is basically biting off more than what can be done. Inventory sharing between servers is something I see as needing to be done step by step.

First step I would think of for sharing inventory is like crossfire inns that you are basically walking out of the main world and into a world built from your personal inventory owning to that world that only you as a player and administrators can access. If this cannot be made work dependably attempting larger inventory sharing is just going to be nightmare. Of course next step would be to allow players to invite other players into their home pockets who cannot in fact change anything. Basically long progressive process that will give mods and other things time to catch up with the changes.

oiaohm on 26 Jul 2020

https://github.com/minetest/minetest/issues/4428#issuecomment-663274337
Seems like a perfect solution tbh

niansa on 26 Jul 2020

http://crossfire.real-time.com/

Just linking the homepage of the game doesn't describe how its cross-server stuff works. Could you at least link some documentation that describes everything needed to understand your post?

When it comes to cross server messaging you have to allow for a bit of lousy.
Performing voodoo/summoning across servers get painful to implement and could be path to exploits.
If you look at crossfire you have another choice. 1 house on the map is everyone house. Inserting a 3d space in and out of a map could get too messy if that is too messy.

Or you can pick a system that can handle these cleanly, like the one-server solutions suggested in this thread.

So I would have inventory only stay if its compatible and it makes sense.

Makes sense to me. It seems though that it could be complicated how you track which groups of worlds should share inventory or other state, though, and also how to handle when one world in a group of compatible ones changes mods/settings.

Now have a door that basically lets a player step into a small world that is hidden in his/her inventory and out of the main world.

So a new server would have to be started for each player who creates a pocket house?

Overall it seems like you are saying that many things won't be possible if Minetest does it how Crossfire does (compared to other suggestions), and that modders should just work around it. Your post seems like a better argument against than in favor of doing it the way you describe. You also haven't provided any technical details about how custom cross-server actions / state from mods will be handled, or how it would be exposed in the modding API.

I like rubenwardy's suggestion but I haven't thought deeply about it.

raymoo on 26 Jul 2020

The engine would allocate a virtual base position for a dimension, for example, { x = 45000, y = 0, z = 0 }. Mods can continue to use old position functions without needing to know about a new co-ordinate dimension, and will be able to index all positions.

Possible. Remember that node hashes may rely on 16-bit coordinate limitation, and also need to fit in 56 bits (effective integer size in Lua, as it’s double internally; strings don’t seem to be good hashes); thus keeping 16 bits for local coordinates we have 56−3×16=8 bits at most for the world index. Well enough for many use cases but not a plenty of space actually.

Versioned APIs - A mod would declare an API version in mod.conf, and the mods function environment would be set to it.

How could that work?

An environment for each dimension, with all mods initialised in it.

Possible. But there is a problem: mods need to store data, and that data may be position- or player-depended, or both (per-player teleportation targets?). That won’t work as-is.

numberZero on 6 Aug 2020

Position hashes would be per-dimension on the C++-side. Not breaking the Lua API for position hashes on the Lua API side could be tricky.

rubenwardy on 6 Aug 2020

Minetest: Different "Dimensions" / Other worlds

Problem

Solution : Different Dimensions

API changes needed

Problems to deal with.

Profits

Most helpful comment

All 184 comments

Related issues