Minetest: CSM causes security issues

[tenplus1]

Since using Minetest 0.4.16 stable on Xanadu it has been noted that quite a few players are exploiting this feature to cheat on server by running bot-chat-text, opening ANY chest to steal items and of course finding ores without the need for digging... Something has to be done to stop this as it takes the fun out of the game entirely and has players complaining of stolen items...

Personally I would like to see this feature disabled entirely but am aware of specific switches available on 0.5.0 to disable certain components of this, which sadly can be re-enabled by pre-compiling your own minetest client with the changes reset...

Server security is more important,

[Ezhh]

Given the issue with chests, saying someone needing to compile their own client is enough of a barrier is nonsense. It would only take one dedicated person with this knowledge to ruin a server's economy overnight, and to upset a huge number of players.

[MarkuBu]

Fully agree. I tried ore detect on a server just to test if it works and it was possible to find enough diamonds for a full armor and a full set of tools in no time.

I never used it and the items are still hidden in a chest, but that doesn't makes fun, especially if some players try to find others chests to loot them

[ghost]

As you say by yourself, it is impossible to prevent cheating because the client is open-source. So if someone really wants to cheat it’s always possible to change the source and compile a modified client.

But you are right. CSM in the current state is an invitation to cheat the shit out of Minetest servers. With 0.5 (or maybe a rushed 0.4.17 release a few days earlier) there will be “flavours” (which should be spelled “flavors”, but that’s another topic ...)

https://github.com/minetest/minetest/blob/f7733f41ea36e7d4531fea61a4cd4b679514d263/minetest.conf.example#L1347L1355

This allows server owners to reduce the amount of information that will be accessible for CSMs. By far not for anything and by far not secure but it’s a good start.

[MarkuBu]

@Ezhh I don't think this is really an argument. Of course, a lot of people can compile their own client and share it, but EVERYBODY is able to install a client side mod, even without any knowledge of coding and compiling.

[Ezhh]

@MarkuBu Sorry, but not follwoing what you mean. 0.5 will introduce flavours which block CSM, but the check is client side, therefore -my comments above-. This doesn't mean I'm in favour of the current situation where everyone can (ab)use CSM freely without even needing to compile. Needing to compile to abuse is clearly better than it currently is, but I feel even that isn't enough of a barrier, at least when it comes to a player being able to help themselves to the content of all locked chests. It only takes one person doing this to really mess up a server.

[rubenwardy]

bot-chat-text

This is very annoying, and send_chat_message shouldn't have been merged. It's totally unnecessary. It's still worth adding server side preventions to this - stopping duplicate messages - as it helps against copy and paste spam.

opening ANY chest to steal items

This is actually a good thing that has come out of CSM - it's exposed security issues in mods. The mods which add these chests need to be updated.

finding ores without the need for digging

This is rather irritating, and there should have been defences put in place before get_node was added. In 0.5.0 you'll be able to disable get_node, however it would be good to also hide certain nodes from get_node - ie, make gold ore appear to be stone when calling get_node.

[stujones11]

Bugs aside, I personally think this whole CSM cheating nonsense has been blown way out of proportion. The worst anyone can really do is detect some ores, which is still possible on older clients using an x-ray texture pack anyway, as for players spamming the chat, simply ban them...

I do agree, however, that CSM was added before it was ready but once the 'flavours' are in place then servers are at no more risk than they were before with hacked clients. I do not think that CSM lowers the barrier to client hacking by any significant degree, cheats like 'fly' and 'ore-detect' are simple enough anyway if you are able to disable CSM flavours and compile the client for your hardware. The latter alone rules out about 99% or the userbase :P)

[tenplus1]

@stujones11 - currently I can connect to ANY server and open ANY chest and take items using CSM, it's not secure.

[stujones11]

@tenplus1 Like I say, bugs aside. That is clearly a bug and like a few other much more serious ones, would probably not have been discovered if not for CSM.

[paramat]

Recent discussions on IRC show a new CSM issue is needed. Many well-respected server owners are very unhappy with CSM, to the point of considering making their servers whitelisted, or even more extreme actions.

Since using Minetest 0.4.16 stable on Xanadu

It was a big mistake to release it for 0.4.16 stable before any restrictions were added. Personally i left the CSM devs (nerzhul, red-001) to the task and didn't monitor what was happening, i didn't really understand CSM, when i realised what CSM actually meant it was after release, i then as you know made a big fuss which made me rather unpopular with some, but led to CSM flavours being added.

Personally I would like to see this feature disabled entirely

I expect you actually mean you want CSM completely removed? As long as the code for client-provided CSMods exists hacked clients can use CSMods and override any server option that tries to disable them.
Maybe a solution is to remove the ability to have client-provided CSMods and only have server-provided CSMods? This may raise the amount of hacking needed, however this is just a guess and may not help much.

The CSM 'flavour' restrictions coming in 0.5.0 are a way to disable certain parts of CSM, such as node getting, anything that can affect a server. The list of CSM functions that can be disabled is very small, because only a few functions affect servers, however any function you feel you want to disable should be requested, that was the plan, add to the list as people request.
I did at first argue that CSM flavours should be able to disable all CSM functions, whether or not they affect a server, but the author was not happy with this (see the CSM flavours PR thread).

but am aware of specific switches available on 0.5.0 to disable certain components of this, which sadly can be re-enabled by pre-compiling your own minetest client with the changes reset...

Being able to hack around CSM flavours is not realy something that makes it irrelevant or useless.
While i mostly agree with your opinion, the obstacle to cheating is not that much lower than before CSM, you still need to alter and compile a client, and things like fly and noclip were easy to hack before CSM. However yes CSM makes more cheating possible with less hacking required.

The worst anyone can really do is detect some ores,

That's not a trivial ability.

[paramat]

0.5 will introduce flavours which block CSM, but the check is client side,

It's a server setting and is not 'clientside', but yes a hacked client could ignore the setting sent to it.
I'm doubtful there is a secure way for the server to prevent a client using a CSMod by hacking, apart from removing the code that allows the client to use CSMods.

Before and after CSM, the only completely secure defense against hacked clients is serverside monitoring and validation of client actions, something we have already in basic form (detection of overly-fast movement etc.). Someone is or was working on developing this further ('Serverside movement' PR). It's a big and difficult task though, but also high priority.

[Ezhh]

the obstacle to cheating is not that much lower than before CSM

Though I agree with almost every word you say above, I disagree on this point. Before CSM, though these things were possible, the bulk of the code was not there in front of people to just take. Instead of writing the full cheat themself, all someone will need to do with flavours is work out how to re-enable CSM for their client. This reduces other cheats to the same level as enabling fly, which people have admitted for a long time is very easy (even I can manage it). And the chest issue, for example, is a lot more serious than someone being able to fly.

I did at first argue that CSM flavours should be able to disable all CSM functions, whether or not they affect a server, but the author was not happy with this (see the CSM flavours PR thread).

To repeat something I have said on IRC: CSM clearly only has a role to play with regard to servers. (If you want to do something on singleplayer, you could make it a regular mod anyway). But the server owner is the one managing and paying for that server; therefore why should they not decide what the player experience on that server is? (If we think server owners are evil tyrants who we need to create ways for players to assert their preferences in spite of, we have bigger problems!).

[paramat]

@red-001 any comment? I read you may be working on server-provided CSMods, which is the more useful part of CSM and the original plan (the MT TODO page in the wiki doesn't even mention client-provided CSMods).
Not that this will solve any of the problems though.

My opinion is that either server-provided CSModding is added or CSM is completely removed. Client-provided CSModding is rather boring and limited and doesn't justify the problems caused by CSM.

[paramat]

This reduces other cheats to the same level as enabling fly

Yes good points, i have probably understated how far the barrier has been lowered.

[paramat]

I am requesting that CSM flavours allows all CSM functions to be disabled, whether they affect the server or not. i won't ask nerzhul because he will not be happy with it, someone else will have to (maybe i'll attempt it).

Despite server-provided CSM obviously being very useful, there is a part of me that wonders whether CSM should just be completely removed before 0.5.0, and re-assessed, since the implementation was a mess. It seems to have been added because it was intellectually satisfying and the possibilities were exciting, without really realising the affect on servers or whether server owners would be happy with it.
I suspect that if server owners had realised the consequences many would have tried to stop it.

[stujones11]

Removing CSM will not remove cheaters, Hacked clients will still exist without CSM the only 'high priority' here is getting 0.5.0 released so the servers can update.

[Ezhh]

@stujones11 We haven't said it will stop cheaters. The problem is how much the barrier to cheating has been reduced compared to before.

[sfan5]

I am requesting that CSM flavours allows all CSM functions to be disabled,

Player: 「Can I have client-side ignoring of certain users in chat?」
Server Owner (in background): 「>:( no, muh safety is at stake」

that said, disabling CSM entirely is massively overblown and circumvented even more easily (unless you sign the builtin Lua code)
instead focus should be put on the features of CSM that cause server owners problems, of which few remain currently

[stujones11]

The problem is how much the barrier to cheating has been reduced compared to before.

I really don't think so, how many of your players are even able to compile the client? If they're using windows or android (which most will be) I doubt there will be many :^)

[tenplus1]

@stujones11 - that's the thing, it takes 1 person to compile a hacked client and release it for free for kids to play, e.g. hacktest.

[stujones11]

@tenplus1 That has absolutely nothing to do with CSM, you have just proved my point :)

[tenplus1]

e.g. to compile a hacked client that re-enables the CSM even if it's disabled server side.

[Ezhh]

I, and other server owners, have had enough people signing in with fly/noclip enabled for themselves. Fly/noclip doesn't break the game (in any major way) and generally just upsets a few people until a moderator gets in game to verify they really are using these abilities. They are also highly visible cheats that tend to get noticed unless the player is very careful. Cheats like the chest issue are much more serious - you only need one player to do this to potentially create huge problems on a server that are very difficult to straighten out again afterwards.

This will become as easy as giving fly to yourself. (It's currently even easier.)

That is the problem.

[stujones11]

e.g. to compile a hacked client that re-enables the CSM even if it's disabled server side.

Why bother, just release a hacked client that has 'all the cheats' :P)

[stujones11]

@Ezhh Fly and noclip are not CSM cheats, afaik

[sfan5]

Cheats like the chest issue are much more serious - you only need one player to do this to potentially create huge problems on a server that are very difficult to straighten out again afterwards.

I'm just going to quote @rubenwardy verbatim:

This is actually a good thing that has come out of CSM - it's exposed security issues in mods. The mods which add these chests need to be updated.

[Ezhh]

@stujones11 I haven't said they are. I think you're not reading what I say.

@sfan5 I agree it's good that we learn about these issues. We haven't once disagreed that it's good to learn about them to fix them, but there's a bit more than that to all this.

[Fixer-007]

Restricting CSM will not fix some of those problems, for example, ore mining, fly, noclip, xray still works without CSM, as well as that taking items out of chest (mods need to update, that is known sec. vuln.). Minetest needs effective nofly, noclip, basic checks on actions, and of course csm restrictions (that are already been implemented anyway). Do not shoot the CSM.

[Fixer-007]

currently I can connect to ANY server and open ANY chest and take items using CSM, it's not secure.

Have you checked on up to date servers? That problem was fixed for the mtg (iirc) and most of the chest mods.

[sfan5]

basic checks on actions

I like how everytime cheating comes up "basic checks" are suggested acting as if Minetest does not have any,
despite the fact that (some of) such checks were present in the very first release of the 0.4 series: 0.4.0
Yes, 5 years ago.

[Fixer-007]

It does not, remember that inventory take/out debacle? And recent one is "No protection against long-range inventory access". And of course CSM has nothing to do with this %)

Or this:

IhrFussel, You can dig nodes that are 300 away from you in Android 3rd-person

[stujones11]

@Ezhh Sorry, I did misread your comment, however, my point remains that the majority of these 'so called' cheaters are not relying on CSM. In fact, many of there players with unauthorized fly privileges are more likely using ropey android clients.

[Ezhh]

@stujones11 Thanks for taking time to re-read.

I know that fly isn't related to CSM. My point is fly is easy to get and, from my experience of running servers I know people do make this change and share it between themselves. CSM has reduced other cheats to the same level of difficulty as fly . Therefore, because we know people do manage to get fly, we know they'll manage to re-enable CSM for themselves even once it's restricted with flavours.

[sfan5]

@Fixer-007
you conveniently omitted the second line when quoting Fussel:

If anticheat is disabled

Nothing to see here.

[Fixer-007]

Stop disabling anti-cheat!

[BluebirdGreycoat]

Some of us are forced to disable anticheat on our servers. The fast-move prevention in particular is a royal pain on a laggy home connection.

And on "Inside the Box" for example, (I believe that server has anticheat enabled) I get yanked around quite a bit as soon as I fly around in edit mode for more than 5 seconds straight. This is due to anticheat's fast-move prevention AFAIK.

Count me among those very upset server owners.

[BluebirdGreycoat]

When I first expressed interest in CSM, I had thought that the mods would be sent from the server and run client-side, NOT that the client would be able to provide its own mods.

That clients are currently capable of supplying their own CSM without respect for the server is absolutely unacceptable. With the current state of things, anyone can make and upload harmful or disruptive mods to the forums or anywhere else, and suddenly we have mobs of newbie hackers. Just like Minecraft.

Oh, and this: http://www.minehacker.org/

[rubenwardy]

Oh, and this: http://www.minehacker.org/

Irrelevant. So is anything related to anti-cheat. This issue is about _csm_

[BluebirdGreycoat]

Not irrelevant. Unless someone can prove otherwise it likely has CSM enabled, and its only a matter of time before it comes packed with _CSM_ hacks.

[BluebirdGreycoat]

So is anything related to anti-cheat.

My comments on anticheat were a reply to Fixer-007, not to the topic in general.

[stujones11]

packed with CSM hacks

What CSM hacks? Ore detection is the only 'cheat' I am aware of. It's not like ores are difficult to find anyway if you dig deep enough.

[rubenwardy]

Not irrelevant. Unless someone can prove otherwise it likely has CSM enabled, and its only a matter of time before it comes packed with CSM hacks.

That fork is a supposedly advanced cheat fork of Minetest, it wouldn't be that hard to add something to search for ores without CSM.

All it takes is for one person to release a client with these modifications, doesn't need CSM to exist. CSM makes it easier to keep the client up to date, and not have to compile for a bunch of platforms.

[Ezhh]

Ore detection is the only 'cheat' I am aware of.

Being able to open locked chests to take items freely feels like a bit of a cheat to me.

[rubenwardy]

Being able to open locked chests to take items freely feels like a bit of a cheat to me.

Completely irrelevant as it's not caused by CSM but by buggy mods

[tenplus1]

@rubenwardy - it's not buggy mods to blame, but without going into too much detail and giving it away, csm can open a simple formspec and gain access to any chest in the game.

[Ezhh]

@rubenwardy Nothing changes that CSM makes it much more accessible than it has been. Yes it is good for finding issues to get them fixed, but it is not good for the server owners who have to deal with the problems it makes in the meantime.

[Ezhh]

Also it feels as though the chief way to discredit the concerns about CSM is to misinterpret what we're saying in ways that make it seem we don't understand what it is. Sadly I understand the issue far better than I ever hoped to need to. I know full well that CSM is simply lowering the barrier to do things that could always be done - the problem is that it is lowering that barrier so far.

[rubenwardy]

csm can open a simple formspec and gain access to any chest in the game.

You can read any chests in the game, but you can't take from it if the chest has the bug fixed. Reading from the chests is not very bad as a vulnerability - incredibly minor actually - and can easily be solved server side by supporting mark_as_private to stop the inventories being sent

CSM is simply lowering the barrier to do things that could always be done

The chest vulnerability may have been always possible, but it's perfectly visible. CSM only makes it easier for info leaking (finding ores) and chat spam (as that function was stupidly merged)

[Wuzzy2]

I agree with most of the discussion so far, but then I stumpled upon this:

server-provided CSMods

:scream: :-1: :skull: :disappointed: :angry:

No, no, no, no, nooooooooo!

I find it quite astonishing that such a huge thing is almost mentioned casually here.

This will open a whole new can of worms and security issues. This is a huge red flag here. Please do not ever do this. Please find other ways to implement your desired features.
Basically you're suggesting to fight the problems with client-sided mods by adding even more client-sided mods. You're pouring oil into the fire! :grin:

As far I understand, the security issues with client-sided modding mostly revolve around cheating. This is bad enough.

But with server-provided CSMods, clients will now be at a new level of danger. The current CSM situation will be completely harmless compared against that.
A vulnerability can lead to Minetest clients to execute arbitrary code.
Then there's the possibility of the man in the middle.
Then the server could be rogue and send malicious code.
And there is so much more which could go wrong.

Clients will be forced to “trust” the server and the connection not to send any malicious code. Besides, who's to say that this server-provided code isn't proprietary software?
With such a channel it will be very easy to sneak in proprietary software and it will be hard to verify the code. Remember the JavaScript Trap (https://www.gnu.org/philosophy/javascript-trap.html)? For Minetest, it would be the same.

Just look at the web: If you're just a bit familiar with the Web, you should know what a security hellhole it actually is. XSS is a daily routine. Please do not make the same mistake.

---

Can I please get a promise from the dev team that server-provided code which clients must execute will never be a thing in Minetest?

---

Quote from paramat:

Despite server-provided CSM obviously being very useful, there is a part of me that wonders whether CSM should just be completely removed before 0.5.0, and re-assessed, since the implementation was a mess. It seems to have been added because it was intellectually satisfying and the possibilities were exciting, without really realising the affect on servers or whether server owners would be happy with it.

:+1:
I think the biggest mistake about the CSM thing as such is the failure to involve the community beforehand.

[BluebirdGreycoat]

@Wuzzy2, that's mostly FUD.

The internet is a security hellhole because it is a huge sprawling mess, and because Javascript and the other various technologies are themselves a huge sprawling mess. NOT because the theory of server-sent code is inherently harmful. Internet security came about practically as an afterthought.

Also, please do not bring the whole proprietary vs GNU into this. It is not relevant to the issue of security.

Please do some research about Lua before concluding that it cannot be used securely, I am not going to argue the details of that here. Suffice to say that given an empty sandbox, the worst a server can do to your client is make it freeze due to an infinite loop. In which case you restart it, see?

[BluebirdGreycoat]

Also, most attacks online require the user to do something (social engineering). Not necessarily a real vulnerability in software. I'm pretty sure that if some server, somewhere, makes a formspec pop up that says "put your credit card # here", NOBODY is going to bite. And as I said elsewhere, badly behaved servers can be removed from the server list.

[raymoo]

@Wuzzy2 A sandbox can be constructed that is as secure as the underlying Lua VM and the API exposed to the mods is. The first would be a new risk, but the second is a risk that already exists since the client receives formspecs, HUD elements, etc. from the server and does stuff in response to it. As for luajit, I don't know of any sandbox-escaping exploits. But to be on the safe side, it may be a good idea to use non-JIT Lua on the client, since JIT compilers present more ways arbitrary code execution could happen.

MITM attacks would still potentially be a problem. Even if they can't execute arbitrary code on the client, they could do things the user doesn't want on the server they are trying to connect to, or send scripts that cause the user to take actions they don't intend. This isn't a new risk though, because a MITM attack could already do things like mess with the user's inventory, use items unintentionally, etc. if it is "proxying" between the client and server.

[paramat]

Although i mentioned personal thoughts of 'removing CSM', i think that would be too extreme and an overreaction.

Can I please get a promise from the dev team that server-provided code which clients must execute will never be a thing in Minetest?

This was the original plan that has been in the MT wiki TODO for years, so i expect is fine with celeron55 and is known to be safe enough by those who know. The server-provided aspect is the truly useful part, without it, as i wrote, we are better off without CSM, because the client-provided aspect is the larger problem with the least useful mods. The TODO doesn't even mention the client-provided aspect.

So i think that the server-provided aspect must be developed to justify CSM, @red-001 has stated it was always going to be anyway. But also, i think we also need to make a big effort in responding to the many genuine concerns, in any way we can. I think that CSM flavours should be extended in the way i requested in the flavours PR thread, and i may work on this.

I think a server should be able to disable any client-provided CSM function, whether or not it affects the server. If a certain feature should not be server-restrictable then we can add it into the core client code, this way all features that clients are free to use how they wish have been agreed upon by the dev team, this is how it has always been.

Quoting celeron55 again https://github.com/minetest/minetest/issues/5915#issuecomment-306424066
"The design of Minetest is to give all possible control to servers."

Once server-provided CSM is added we could then possibly (depending on the situation and problems caused) consdier removing the client-provided CSM ability, as this is the more problematic and less useful part not mentioned in the TODO, which possibly does not have the support of @celeron55

[ghost]

Once server-provided CSM is added we could then possibly (depending on the situation and problems caused) consdier removing the client-provided CSM ability, […]

That would be perfect as long as it is possible to disable server-provided CSM in the client and as long as a permission-based security setup is in place (like Android for example where apps request permissions on first use).

[rubenwardy]

permission-based security setup is in place (like Android for example where apps request permissions on first use).

What for? We won't be exposing any APIs like http or file access to CSM. The only dangerous thing could be mod communication channels

[ghost]

The only dangerous thing could be mod communication channels

And thus have it as secure as possible with a user-defined whitelist of allowed channels.

[sfan5]

That would be a terrible idea, every time a modder adds a new feature it's not going to work at all for everyone as it's not present in the whitelist. Besides whitelisting individual channels is stupid as you can send whatever you want over a single one.

IMO Mod comm. channels are not a security-relevant feature and do not need separate restrictions.

[ghost]

every time a modder adds a new feature it's not going to work at all for everyone as it's not present in the whitelist.

Learn how the mentioned permissions system of Android works for the end-user.

[raymoo]

We could also follow Android in the specific things they require permissions for. For example, nothing in CSM would fall under an Android permission.

[sfan5]

Learn how the mentioned permissions system of Android works for the end-user.

That's even worse, you're asking users whether they want to allow some technical details they don't understand. Essentially a "click to make it go away" popup as it has been found ineffective for security purposes already.

[ghost]

Yeah, whatever, dude. I know you want this shit in the client. So then do it. Make it insecure on purpoose like the first implementation that was rushed into the code to have it released without any security in mind.

But at least make it completely disable ffs.

[sfan5]

I don't think server-sent CSM would make it into Minetest without an option to disable those, I support such an option myself.

[stujones11]

Regarding server supplied client-mods, no matter how well sand-boxed they are, concerns over security are only natural. I would also like to see an option to disable all server-sent mods but still have the option to install them manually from source.

[ghost]

For example, nothing in CSM would fall under an Android permission.

It’s about the concept (ask before you use XYZ and store the decision). Of course CSM permissions are different from smartphone app permissions. Duh.

[raymoo]

@dsohler I can't think of anything server-sent CSM would do that would need to be barred behind permissions. What I meant is that Android puts things behind permissions because they are actually dangerous.

[BluebirdGreycoat]

@stujones11, I agree that an option to disable server-sent mods should be present. However there should be NO option to install any CSM mods from source. That just opens the whole user-provided CSM issue again. I propose to view the option to disable server-sent CSM as similar to that Noscript addon that people use.

Disabling CSM is like disabling web scripts. Some people do that and we should support it, but we definitely should not allow the user to inject their own scripts into the client in order to communicate with the server. Obviously bad users can get around this, but it should not be made easy.

@rubenwardy said,

The only dangerous thing could be mod communication channels

... And then people start arguing about permissions!

I should point out that IF it is implemented correctly (current dev behavior does not give confidence in that area, I strongly suspect), the idea of CSM talking back to the server should present no security risk at all.

Just think: why would such a risk exist? Obviously, because such communication could be providing the user's private details to the server. But first the CSM must be able to access such information. If it is sand-boxed properly (no access to file-system, no way to communicate except with the current server, no direct access to the graphics card, etc.) then there is no way for CSM to obtain user-data in the first place.

Just concentrate on a secure sandbox, with absolutely nothing supported that is outside the domain of the Minetest client, and the rest of this stuff will fall into place automatically due to its nature.

Obviously, going forward I think the devs should start communicating more throughly with the users, especially as we start moving into the stage of deciding what functions CSM should be allowed to access and what the client should be allowed to do and what it absolutely cannot do.

[BluebirdGreycoat]

@stujones11, however I do agree that the users should be able to verify what the server sends them by downloading it and viewing it themselves. Just that they shouldn't be able to install the CSM themselves.

Such a possibility might be implemented by providing an option, on connecting to a server, to simply download any CSM it sends and then disconnect; thus allowing the user to close Minetest and read the code in their favorite text editor.

[stujones11]

@BluebirdGreycoat User installed CSMs will pose no danger to a server whatsoever once the flavour system is properly in place and (more importantly) the _server-side_ mod bugs are fixed.

[rubenwardy]

I'll disapprove any server-provided mod system which doesn't guarantee authenticity, allow remote disabling, and ideally give reproducibility.

• Authenticity - who wrote the mod?
• Remote disable - can we disable server-provided CSM for all users?
• Reproducibility - if it is reported that a mod causes problems, can we obtain the source code for those mods? This isn't as essential, but would be good. It would also be good to view the code of any production CSM.

Authenticity can be given by either signing the mods, or using a third party mod store. This will allow us to show a dialog like "Modname by author" or "Modname modified by AA, originally by author" without the server lying.

Remote disable can be an endpoint on minetest.net, and should be separate from the serverlist incase the user disables it.

Frankly I don't trust the Minetest engine not to have any issues which could be abused.

[stujones11]

Maybe the answer is that all CSMs, either user or server provided, need to be installed from a trusted source. An integrated 'mod-store' would be the best solution IMO.

[sfan5]

I'll disapprove any server-provided mod system which doesn't guarantee authenticity, allow remote disabling, and ideally give reproducibility.

I've thought about this a bit with the analogy of a web browser:
Would I want my browser vendor to have a remote killswitch to prevent certain scripts from running?
Would I want that all running scripts are first audited by my browser vendor?
No, I wouldn't.

Concerning faulty scripts we already have a (more drastic) equivalent of "Safe Browsing": removing servers from the server list.

Having reliable information on who wrote which CSM mod is nice I guess but brings the whole burden of maintaining a trusted source, which I don't see as viable (and even hindering innovation) given that this is a FOSS project already running low on manpower, especially as such a "trusted" mod store has failed before.

[liberodark]

You could use Github for storage and updating of the store and csm this is possible and also easier and less difficult to manage :+1:

[sfan5]

The problem is not primarily that it's difficult to manage or host, the problem is that there is nobody to review every CSM piece of code any of the numerous modders come up with.

[ghost]

Would I want that all running scripts are first audited by my browser vendor?

I would all scripts are audited (or at least seen as secure because I trust the site) by me, thus I have a Javascript whitelist. Same should be implemented for server-provided CSMs. I only would like them if I trust the server owner and then I would like to select which CSM I actually want from that particular server.

[liberodark]

but if it's in open code there is less risk I think we should favor open source and also allow visibility of users or server administrators

[Ezhh]

If you make the code public somewhere, how to guarantee that's the exact same code the server then sends to the client? If you need to receive it from the server first to check, then it's difficult to review before...

Also for server-provided CSM to be really helpful to servers, the server might not end up being playable without it, so how to handle when someone connects but refuses a CSM mod that is actually vital for the gameplay experience? (This assumes CSM would manage to provide anything vital...)

[liberodark]

it is not difficult to make a simple signature of the files could be enough!
May you be right how to do if a user refuses a mod it seems unthinkable in any game that is possible
Simply check files sha512 is possible

[raymoo]

@Ezhh You could compare hashes with the other source.

If someone refuses a CSM they could just be shown a notice that they will lose important gameplay features or something like that.

[liberodark]

For me it seems important that the user can know if it is good or not and have access

[Ezhh]

@raymoo Of course, but it needs to all be simple for the user. If it's too much work or effort, then I'm sure most won't bother and would just look for a non CSM-using server anyway. The last thing we need is to expect players to jump through more hoops before they can join and play.

[paramat]

@red-001 do you still intend to code server-provided CSM? I expect this may only be properly tested in time for 0.5.1, doesn't have to be done in time for 0.5.0, but maybe it will.

I intend to strengthen CSM flavours, to allow disabling all functions, as has been requested by our most respected server owners.

Then during 0.5.0, with CSM flavours in effect, we can see how much trouble there actually is and go from there. Hopefully flavours will significantly improve the situation.

the problem is that there is nobody to review every CSM piece of code

Very true.

[BluebirdGreycoat]

To much talk, not enough logic. Why is everyone discussing CSM code review? Or a trusted-mods repository? Or signed code? Or remote kill-switch?

Just provide a way for users to disable CSM (like with Noscript in your browser), and concentrate on a correctly implemented sandbox.

Frankly I don't trust the Minetest engine not to have any issues which could be abused.

Fortunately nobody has to, you just have to trust the Lua VM.

Read this please: https://stackoverflow.com/questions/34388285/creating-a-secure-lua-sandbox

A very important point is that the client MUST NOT use LuaJIT! There can be no bytecode allowed anywhere, period. Start with an empty sandbox (this means empty!) and check and whitelist each chosen function individually. IIRC recent versions of Lua make this even easier than before.

A final point I want to make is that CSM should not be enabled by default. There should be a period of a few months (maybe a year) where it is disabled by default, requiring a setting in minetest.conf to enable it for devs/testers/the very brave. This way you can test the sandbox for mistakes over time. A perfect sandbox IS possible but will need work (and research if you dare to include the fancier functions -- which you shouldn't).

[BluebirdGreycoat]

Here's a better link: https://stackoverflow.com/questions/1224708/how-can-i-create-a-secure-lua-sandbox

Also, see this! http://www.lua.org/demo.html

Obviously, SOMEBODY is confident enough to let just anyone use that.

[paramat]

client MUST NOT use LuaJIT!

A good idea anyway as it is troublesome and is not reliably maintained.
Do we currently allow LuaJIT for CSM?

[BluebirdGreycoat]

I have not looked, but it would not surprise me if the client does use LuaJIT. The server does, and they share source code.

This will have to be fixed before server-sent CSM can be considered, obviously.

[BluebirdGreycoat]

On sandboxing: also (rather obviously), the server-sent CSM should not use the same Lua state as the main-menu Lua state. Otherwise a server will be able to corrupt the client's main-menu (even if that Lua state is sandboxed, which it certainly is not).

Edit: the client doesn't use the same Lua state for main-menu and game scripting after all.

[Wuzzy2]

Amazing how almost everyone just assumes that server-provided client mods will come definitely. We no longer discuss if this is a good idea in the first place. WTF?
Why did almost everyone ignore my post?
I think it's time to get out of La-La Land and face reality.

This time I try to summarize:

• Any vulnerability in the client will lead to remote code execution on the freaking client! I can not stress this enough! This is a kind of vulnerability which never happened before.
• A malicious server operator could send malicious code
• A man in the middle could inject arbitrary code
• A script can be buggy itself and cause damage
• The player is forced to trust a lot of code and people (MT is played by children, too. Do you really expect them to know who to trust?)
• The code complexity will completely explode. And making complex code secure is extremely hard. Please be honest: Are you really dedicated enough to handle all this?
• Given the track record of past CSM development, I have to expect that the will be serious bugs and design flaws. None of the core developers has provided evidence to be skilled in writing security-critical software
• It's very easy for servers to sneak in proprietary code
• It's very easy for servers to sneak in spyware
• You would have to audit the relevant code. And you need very competent people for this.
• All of this will cost a huge amount of precious dev time
• Worst of all: The benefits don't even sound that awesome

And finally: Why do you want server-provided client mods so badly? Seriously, I don't even understand why you consider this to be so incredibly important that you push it at all costs. I don't think this feature is even great or important. All dev time sunk into this will be stripped away from making the engine more awesome and fixing important bugs.
No feature can possibly so awesome that you really can ignore all these HUGE issues. Why am I the only one who flatly rejects the notion that servers have to send mods to the clients?

All the answers I got so far are really disappointing. You're all dancing around the real issues by offering fake solutions:

• Sandboxes can be buggy and will be escaped. We can't trust them blindly.
• All trust and authentification measures and checksums will be useless if the server operator themselves (which you probably trust by default) is acting maliciously.
• Not using LuaJIT is a no-brainer, but doesn't address ANY of the core problems.
• Implementing a NoScript-like architecture is just insane. This is just moving your responsibility as a coder away to the user. A terrible idea.
• Signing mods just means that you have to trust the person who signs it. Because we all know how incredibly trustworthy Certificate Authorities are!!1!
• Metadata, like author of the mod, can be faked and even if not, that doesn't prevent anything
• Making this feature “optional” for clients is just fooling yourself. There will be servers which give you no other choice than to accept the client mods, otherwise there will be bugs, you will be kicked or whatever. So in reality, players won't really have a choice at the end and the feature is really not optional at all. If disabling this feature means you're locked out of 75% of servers, then you don't really have a choice

At the moment, I completely reject server-provided client mods and I think it is an insane waste of time. Even if “optional” (because I'm not stupid).

[paramat]

everyone just assumes that server-provided client mods will come definitely

That has been the plan in MT TODO for years, it has been discussed by celeron55 and other devs and the consensus is it is possible to do safely, they probably know best.
We are discussing whether server-provided CSM is a good idea, many of the posts here do that.

Why do you want server-provided client mods so badly?

It's the only truly useful part of CSM, the client-provided aspect wasn't in the original plan.

at all costs

We won't ignore issues if there are issues, if it's a bad idea it won't be done.

[BluebirdGreycoat]

Welp, here goes.

Any vulnerability in the client will lead to remote code execution on the freaking client!

Strictly speaking this is already possible. And, still strictly speaking, a sandboxed Lua state does not make it easier, because the functions in said state are not the functions an attacker is probably looking for. They'll want access to libc or its equivalent.

A malicious server operator could send malicious code

See sandbox.

A man in the middle could inject arbitrary code

Already possible, you don't need Lua for this, you just need a bad pointer somewhere. But in the event that the code injected is a Lua script and intended for the VM, then in that case, see sandbox.

A script can be buggy itself and cause damage

Buggy scripts crash with an error message. A sandbox is designed to take care of those which do not crash, and which are malicious w.r.t the integrity of the user's data. The remaining types of bad scripts generally fall into 2 categories: a) memory hogs, and b) never-ending loops. Neither is critical, merely annoying. 'A' can be solved by imposing memory limits on the VM. 'B', due to its nature, can never be solved (and is easily doable in any browser. And yet, I don't know of any websites that exploit it).

The player is forced to trust a lot of code and people (MT is played by children, too. Do you really expect them to know who to trust?)

The whole point of a sandbox is to reduce (greatly) the need to trust. Also, you already trusted Minetest by playing it (and see previous points). Are you SURE the client-server communication protocol does NOT contain any buffer overruns? Keep in mind that Lua is a well-tested and well-used language. It is very unlikely to have any bugs of that sort, unlike the custom protocol that Minetest uses.

The code complexity will completely explode.

Have you looked at the sandboxing examples for Lua? Hardly complex.

Given the track record of past CSM development, I have to expect that the will be serious bugs and design flaws. None of the core developers has provided evidence to be skilled in writing security-critical software

You do have a point here, I admit. And the only way to solve this is to review the code. Obviously I'm not a dev but I will nonetheless be one of those looking at the sandboxing portions of the code if/when this becomes reality. Keep in mind that I play Minetest too, and there is plenty of personal data on my machine.

It's very easy for servers to sneak in proprietary code

That is purely political. Nothing to do with security. Also, you accepted proprietary (presumably) code on your machine when you came here to post.

It's very easy for servers to sneak in spyware

See sandbox. How is a bad Lua script gonna give my info away if it can't get it? Your browser is more at-risk by far due to a) Javascript, and b) massive API.

You would have to audit the relevant code. And you need very competent people for this.

To be more precise, you need people who can read and understand documentation. Lua is designed to be sandboxed, and there is enough documentation on the subject. As long as nobody writes code that looks like it belongs on https://codegolf.stackexchange.com/ then auditing it is not hard, as long as you can understand documentation.

All of this will cost a huge amount of precious dev time

In terms of development: no. It's the review of the sandbox that will take awhile (you'll notice elsewhere that I said the review and testing should take a few months to a year). But once that is done, there is no further need to review it unless a change is made.

Worst of all: The benefits don't even sound that awesome

If I really put my mind to it, I could probably list more valid uses for server-sent CSM than there are posts on this page. But seriously, just look at the internet. What would it look like if everyone was forced to go back to pure HTML and CSS?

Sandboxes can be buggy and will be escaped.

Do you have any real-world examples of someone escaping a Lua sandbox using a technique that did not involve bytecode or the presence of a function that shouldn't have been whitelisted? Because all the sandbox escapes I find in my own searches involves either bytecode, or the use of some arcane library, or because somebody forgot to remove the package table even though they removed other tables.

All trust and authentification measures and checksums will be useless if the server operator themselves (which you probably trust by default) is acting maliciously.

True, which is why I don't support that. I support sandboxes instead.

Not using LuaJIT is a no-brainer, but doesn't address ANY of the core problems.

That is an implementation detail of any proposed sandbox. Note that there are 3 core problems: bytecode, a bad function that shouldn't have been in the sandbox, and user error. Bytecode is the first and greatest problem. Not using LuaJIT, and not loading bytecode, solves it. Only 2 problems left, see?

Implementing a NoScript-like architecture is just insane.

Having one is a matter of principle. I support the option because I do not support forcing people to do what they don't want to do.

Signing mods just means that you have to trust the person who signs it.
Metadata, like author of the mod, can be faked and even if not, that doesn't prevent anything

Agree. And if we have a sandbox then we don't need that.

Making this feature “optional” for clients is just fooling yourself. There will be servers which give you no other choice than to accept the client mods, otherwise there will be bugs, you will be kicked or whatever. So in reality, players won't really have a choice at the end and the feature is really not optional at all. If disabling this feature means you're locked out of 75% of servers, then you don't really have a choice

I think you just described the internet.

[Wuzzy2]

Any vulnerability in the client will lead to remote code execution on the freaking client!

Strictly speaking this is already possible.

This doesn't sound encouraging.
So because it's insecure, it okay to increase the attack surface?

And, still strictly speaking, a sandboxed Lua state does not make it easier, because the functions in said state are not the functions an attacker is probably looking for. They'll want access to libc or its equivalent.

I think you're being naive that the sandbox solves everything. The sandbox must be super-safe and must never ever have any bug. A single critical bug can be catastrophic. And I simply find it hard to believe that the sandbox as implemented in Minetest will be safe.

Sandboxes can be buggy and will be escaped.

Do you have any real-world examples of someone escaping a Lua sandbox using a technique that did not involve bytecode or the presence of a function that shouldn't have been whitelisted? Because all the sandbox escapes I find in my own searches involves either bytecode, or the use of some arcane library, or because somebody forgot to remove the package table even though they removed other tables.

So what? Even if Lua's library is perfectly safe, the Minetest code which adds this library may still be buggy.
Also, I shouldn't be required to proof that the sandbox is unsafe. The developers should proof that it is safe.

But seriously, just look at the internet. What would it look like if everyone was forced to go back to pure HTML and CSS?

I find it hard to take someone serious who does not know the difference between the Internet and the Web.

Anyway: I'm not really a friend of JavaScript either. I think the architecture of the WWW is a big ugly mess.
It's not something to draw inspiration from.

Most of the JavaScript code in the Web is completely unneccessary.

That is purely political. Nothing to do with security.

This point should be addressed nonetheless. Minetest is free software, almost all mods for Minetest are free software, so I think it is perfectly reasonable to bring it up. Users might easily be “duped” into thinking they use 100% free software and they don't even realize that the server they just connected to sneaked in propriertary code … This is very fishy.

You would have to audit the relevant code. And you need very competent people for this.

To be more precise, you need people who can read and understand documentation. Lua is designed to be sandboxed, and there is enough documentation on the subject. As long as nobody writes code that looks like it belongs on https://codegolf.stackexchange.com/ then auditing it is not hard, as long as you can understand documentation.

Nobody stepped up so far.

All of this will cost a huge amount of precious dev time
In terms of development: no. It's the review of the sandbox that will take awhile (you'll notice elsewhere that I said the review and testing should take a few months to a year). But once that is done, there is no further need to review it unless a change is made.

No matter how you call it, at the end it will still take a lot of time.

Implementing a NoScript-like architecture is just insane.

Having one is a matter of principle. I support the option because I do not support forcing people to do what they don't want to do.

And how on Earth do you make this system usable and understandable for the average player (including kids)? That something like NoScript is even needed is already a design flaw in my opinion. Why should players even be bothered with something like this in the first place? Make the system so that a NoScript-like thing is obsolete.
Also, NoScript for Minetest would be useless if it locks you out of a lot of servers.

Worst of all: The benefits don't even sound that awesome

If I really put my mind to it, I could probably list more valid uses for server-sent CSM than there are posts on this page.

I don't believe you. I'm not just talking about any use cases. These use cases should be use cases which would also be impossible with other solutions, such as simply making the net protocol smarter, improving client prediction, and so on.

The code complexity will completely explode.

Have you looked at the sandboxing examples for Lua? Hardly complex.

It's not just about the freaking sandbox! xD
It's also about everything else. The complexity explodes because now you have to maintain two Lua APIs. Maintaining just one Lua API is already time-consuming. Modders now also have to deal with client scripts, which is a complete new world. The amount of code will explode. The amount of code outsourced to mods will increase even more, while the engine will be (relatively spoken) dumber and dumber. Mods will inevitably contain a lot of bloat and implement the same feature over and over again.
I don't like minimalist APIs which lack even basic functionality so scripts are forced to re-invent the wheel.
I like it more if core problems are solved in the engine and net protocol as often as possible possible without the reliance on script. The benefit is less code, less bugs in scripts and a shared common base.
And I simply don't see a problem which is really ONLY possible to be solved with server-sent scripts.

This point is not really security-relevant, but one of my biggest problems with the idea.

[sfan5]

Given the track record of past CSM development, I have to expect that the will be serious bugs and design flaws. None of the core developers has provided evidence to be skilled in writing security-critical software

I think you're being naive that the sandbox solves everything. The sandbox must be super-safe and must never ever have any bug. A single critical bug can be catastrophic. And I simply find it hard to believe that the sandbox as implemented in Minetest will be safe.

While not equivalent to sandbox, the mod security has been bug-free (so far) since the day it has been added to Minetest.

The developers should proof that it is safe.

Mathematical proof of safety is impossible at this level, especially when "unsafe" languages such as C or C++ are involved. (Not sure what kind of different proof you want.)

This point should be addressed nonetheless. [...] Users might easily be “duped” into thinking they use 100% free software and they don't even realize that the server they just connected to sneaked in propriertary code … This is very fishy.

Just because this is a FOSS project, doesn't mean Stallman has to be followed by the word.
Continuing the analogy with web browsers: Very few people see a conflict between free software (e.g. Firefox) being used to run propriertary code, and the ones that do are already using LibreJS

Nobody stepped up so far.

Code will be audited by core devs as with every PR.
It is correct that more care needs to be taken when reviewing security critical code, but this hasn't resulted in any problems in the past.

And how on Earth do you make this system usable and understandable for the average player (including kids)?

You don't. Have you seen children caring about software licensing? Me neither.
Just as with a "NoScript" browser addon, such functionality is for users that would like to have more control over which code they run and that understand the concepts behind CSM and server-provided mods.

The complexity explodes because now you have to maintain two Lua APIs. Maintaining just one Lua API is already time-consuming.

Maybe you have missed it, but CSM does already exist and with it the "two Lua APIs" you claim to be so disastrous for everything that ever touches it.

The amount of code will explode. [...] The benefit is less code, less bugs in scripts and a shared common base.

ｂｌｏａｔｗａｒｅ
How is this relevant at all? Nobody is suggesting to move core functionality to CSM.

[ghost]

Why do you want server-provided client mods so badly? Seriously, I don't even understand why you consider this to be so incredibly important that you push it at all costs.

Quoted for emphasis and question.

I don't think this feature is even great or important.

It isn’t. There is nothing a server-provided CSM could solve that couldn’t be done in core or with regular server mods.

[BluebirdGreycoat]

@Wuzzy2: I think it is clear by now that you actually have not researched sandboxes for yourself. Nor do you appear to have experienced for yourself how much easier they make development.

So because it's insecure, it okay to increase the attack surface?

A sandbox does not increase the attack surface. It reduces it in the long term. Without a sandbox, the devs would eventually have to write every possible feature (an endless number) themselves and include it in the network protocol. That is far, far more code (and possible bugs) than just writing a secure sandbox. A sandbox is far cheaper to verify than all that extra code the devs would have to write in order to implement the features manually.

A single critical bug can be catastrophic.

Correct. Your error is in assuming that verifying the sandbox is hard. I did not say it is hard, I merely said it would take time, and that time is mainly to let people test it and build confidence.

I find it hard to take someone serious [...]

A cheap and inaccurate shot, that was.

Make the system so that a NoScript-like thing is obsolete.

That's exactly what a sandbox does. NoScript is for people who don't trust their sandbox, of which kids are probably not a part. (And in the world of the internet, that distrust has been earned.) It's also so that users will have a choice. I find it rather odd that you are against users having a choice in the form of a simple option to accept or reject server-sent CSM. I also find it odd that you seem not to have noticed that I support having server-sent CSM be disabled by default during the testing period. In other words, I support _opt-in_. One of the problems with the internet was that Javascript wasn't _opt-in_. You had to opt-out, and most people didn't do that; the result was that scripts spread like wildfire before the issues and costs were properly understood. We generally understand the issues now. By requiring the user to opt-in with a hidden option, the devs can drastically slow the adoption of SS-CSM, thus allowing for that long period of testing. If an insurrmountable problem comes up during that period, it will not cost the devs too much to simply abort.

Also, NoScript for Minetest would be useless if it locks you out of a lot of servers.

Let's wait for empirical data before blindly believing that most servers will actively kick players who refuse CSM. There is a big difference between being _kicked_ and continuing to play without advanced features. Many servers will probably support both, actually, just as most websites still function without Javascript, though with a lot of missing stuff. It is not a binary choice as you insist on saying, it is a sliding spectrum.

Regarding code complexity:

It's not just about the freaking sandbox!

But it is. The sandbox is the only thing that matters. The rest of that paragraph isn't security related.

Users might easily be “duped” into thinking they use 100% free software and they don't even realize that the server they just connected to sneaked in propriertary code

Well, since people with that philosophy (with the exception of RMS) typically do work on the internet anyway, thus allowing proprietary code to run in their browser, I honestly don't take the philosophy seriously. This is something that kids get right by not caring. Whether some software has the GNU badge or the FOSS badge or the OSS badge or the M$ badge doesn't matter. People who write software will either treat your data as private or they won't, and what license they choose does not restrict their choice.

@dsohler said:

There is nothing a server-provided CSM could solve that couldn’t be done in core or with regular server mods.

That misses the point. Of course everything is already "technically doable". The goal is to make it _easier_. See the second point of this post near the top.

[ghost]

Of course everything is already "technically doable". The goal is to make it easier.

How? For who? Any proof? What are your use cases?

[paramat]

It's the review of the sandbox that will take awhile (you'll notice elsewhere that I said the review and testing should take a few months to a year).

Yes, even if server-provided CSM can be coded before 0.5.0 i suggest we don't add it to 0.5.0, 0.5.1 at the earliest after careful testing.

[BluebirdGreycoat]

@dsohler, those are trivial questions. I'm not going to do your research for you.

[ghost]

You claim – you prove.

[BluebirdGreycoat]

To continue w.r.t code complexity:

The complexity explodes because now you have to maintain two Lua APIs. Maintaining just one Lua API is already time-consuming.

We already have two APIs, as @sfan5 pointed out. It's just that the CSM API wasn't supposed to be user-accessible, and that needs to be reversed.

Modders now also have to deal with client scripts, which is a complete new world.

Those people will necessarily be the same people who write server-side mods. Adding server-sent scripts in no way reduces the ability of modders who have less skill to write mods that are wholy server-side. This will benefit the clever modders and will neither harm nor help modders who do not wish, or for some reason cannot, benefit from it.

The amount of code will explode.

Nothing wrong with that, as long as that code isn't in the engine itself. In any case this situation already exists, just look at all the server-side mods we have.

The amount of code outsourced to mods will increase even more, while the engine will be (relatively spoken) dumber and dumber.

This is actually a good thing. A "dumb" engine is easier to secure and easier to verify. The "smarter" the engine tries to be, the less secure it will be.

Mods will inevitably contain a lot of bloat and implement the same feature over and over again.

Uh, what? First I doubt this will be the case, and if it was the case then we'd have that problem already. How many protection mods do we have, currently? And yet none of them are exactly the same. Variety is good.

I don't like minimalist APIs which lack even basic functionality so scripts are forced to re-invent the wheel.

The smaller the API, the better for security and verification. That being said, there is nothing wrong with having mods that overlap or even duplicate functionality. Variety in implementation is a good thing.

I like it more if core problems are solved in the engine and net protocol as often as possible possible without the reliance on script. The benefit is less code, less bugs in scripts and a shared common base.

Solving a problem with hand-written C++ code in the engine is not less code, and is not safer code, than just running a script in a sandbox. Also, bugs in server-sent scripts are a problem for server operators, not the Minetest devs. And we already have a shared common base, that is, builtin and minetest_game to a lesser extent.

And I simply don't see a problem which is really ONLY possible to be solved with server-sent scripts.

That's the wrong way to look at it. Ask anyone who has done development on game engines with client-side scripting (some permit server-sent code, some do not). They'll all probably tell you that the goal is to make development EASIER, and to allow non-C++ coders to get in on the development. Enabling server-sent code in Minetest effectively allows server operators to extend the network protocol without having to wait for the devs. The Lua sandbox is what must be used to make those extensions safe.

@dsohler:

There is nothing a server-provided CSM could solve that couldn’t be done in core or with regular server mods.

You are the one who has made a claim here. An implicit one. Your claim is that implementing, using custom C++ and network code, all new features that require changes to the network protocol, is the same or less difficulty and has greater security than adding a Lua VM and securing it, and thus, since you claim the difficulty is the same (or less), and that the security is probably greater, you say there is no justification in having a Lua VM for server-sent scripts. Just code everything manually, you say.

I don't think you can prove that claim; if you could, scriptable game engines wouldn't be a thing.

My own claim is that adding a Lua VM makes things easier on everybody, in the long term. Compared to your (implicit) claim, my claim is trivial and you can easily prove it or disprove it yourself.

As I said, I am not going to do your research for you.

[ghost]

You are the one who has made a claim here.

I did not made a claim. I just posted my opinion, but you said. “If I really put my mind to it, I could probably list more valid uses for server-sent CSM than there are posts on this page.”. Well, then … Please do so.

If you’re not willing or able to then please STFU about how great and usefull server-provided CSMs would be.

[BluebirdGreycoat]

Civility, please.

There is nothing a server-provided CSM could solve that couldn’t be done in core or with regular server mods.

That's a claim. It's also an opinion but that is irrelevant.

I'm not going to list possibilities because I know that you'll simply respond by saying that all given examples could be implemented in core by dev effort. Thus missing the point completely.

In other words, what I'm saying is that you worded your statement in such a way that you'll always be able to respond to any rebuttal. That tactic is generally used when one doesn't want to actually discuss the details of the issues at hand. So I won't engage it.

[ghost]

I'm not going to list possibilities because I

… cant.

I know. But that’s okay. Keep living in your dream world. Good bye, have a nice life.

[BluebirdGreycoat]

Here is the link to the source used in the Lua Live Demo that's at http://www.lua.org/demo.html.

http://webserver2.tecgraf.puc-rio.br/~lhf/ftp/lua/5.1/demo.tar.gz

Sandbox is for 5.1 obviously. They seem to use a blacklist, interestingly. But elsewhere they say to use a whitelist system. This should be considered canonical proof that the Lua developers are confident in the security of their library.

@dsohler, thank you for proving that you are not interested in the real issue, and for proving that you are only interested in examples which you can refute (and then argue about whether they should have been refuted or not). Sorry I can't bite, I'm busy.

[BluebirdGreycoat]

I also want to point out that Lua is designed to be sandboxed from inside. This is a good thing because some of those functions would be pretty snarly if called from the C++ side. It also means that the devs can use other people's sandboxing scripts on the web as source inspiration and a way to ensure that nothing got missed.

[raymoo]

It isn’t. There is nothing a server-provided CSM could solve that couldn’t be done in core or with regular server mods.

This is a bad argument, since anything can be done in core. You included "regular server mods" in your sentence, but there is nothing special about them if you are saying that something shouldn't be included if it can already be done with something else, since anything in a server mod could be done as part of the engine. The reason there is a modding API in the first place is to avoid bloating core with a bunch of features where each feature isn't needed by that many people. An example is GUI / HUD widgets. Should every desired widget be implemented in core? An example of such a widget might be a power bar that the player has to time their mouse input so that it stops at some marker. I would consider this too specific to put in core, and network latency would make a server mod implementation of this unworkable.

I think the security argument has more merit. As far as I can tell, the following possible security problems have been identified:

• A malicious server exploiting the Lua VM to do something bad (may or may not include escaping a sandbox)
• A malicious server exploiting the CSM API to do something bad (may or may not include escaping a sandbox)
• A man in the middle doing some of the above things

Lua itself:
It is possible that the Lua VM and standard libraries themselves contain vulnerablities that lead to escaping the sandbox, but the vanilla Lua codebase is simple and small, providing less space for exploitable bugs to hide. Lua has also been used extensively for game scripting and as far as I know there have not been any documented cases of sandbox-escaping due to a bug in the interpreter (rather than because some part of the libraries was provided that shouldn't have been).

LuaJIT could potentially allow attacks on the JIT compiler since it allows attackers to write executable data to memory. I would suggest not using it for this reason, but I don't know of any related vulnerabilities for LuaJIT either. Once the code is in memory, the attacker needs to have some way to jump to it, which may be very difficult given OS mitigations.

The CSM API:
I would divide the types of vulnerabilities that could come out of the CSM API into a few categories. The first category is just the API exposed doing something incorrectly or being too loose. An example would be a file IO API that allowed writing to files anywhere the user has access. The remote CSM API should have almost none of this kind of API anyway, so this shouldn't be a problem for it. The only thing I could think of is if remote CSM has access to a settings API, it may be able to write to settings it shouldn't have access to, if the API does not do proper checking. Hopefully the code for this would be simple enough to audit, but if not, that could mean that remote CSM should not have any API that requires any filesystem / database access. It's important to note that dofile works on files, so should only have access to some memory-only "virtual filesystem" (as people have suggested already).

The second category is exploits coming from memory bugs or other bad things. This isn't specific to remote CSM and could occur anywhere, so it is a possible consequence of any client-server interaction. I won't address it since I think the posters agree that we shouldn't just have no features.

The third category that I think is relevant is any bugs that, through interaction with Lua, somehow exposes a way to escape an otherwise secure sandbox. This could happen if the API allows code to get debug functions that aren't available in the sandbox somehow, or use them indirectly through the API. None of the current scripting API touches these suspect functions though, so I think it is reasonable to believe that this kind of exploit is not a problem.

Men in the middle:
If CSM is insecure against malicious servers, it is also insecure in the same ways against MITM. But otherwise, a man in the middle can only ruin the player's interaction with the server, the same as it is now. So MITM is only really a problem if remote CSM is insecure in other ways already.

It seems to me that remote CSM is unlikely to have any serious exploits that are unique to CSM, given the sandboxing is done conservatively (please use a whitelist). If you have problems with any of my points above, please respond.

[BluebirdGreycoat]

We shouldn't have a settings API, though. All settings the user might choose for their CSM (sent from server) should be stored on the same server they got the CSM from. If settings were stored in the client (like browser cookies) that would open the door to tracking users from server to server.

Also, we don't need dofile or any of its neighbors. I can't think of any reason SS-CSM should need to read a file (much less a script file) on the user's filesystem; all the data necessary should come from the server.

[paramat]

Server-provided CSM is useful because it can provide features that don't suffer from network lag, useful in the way that current clientside engine code is useful. It seems it can do interesting things not doable by a server mod. However i'm not the best person to ask, and i could live without it.

[raymoo]

@BluebirdGreycoat What if a mod wants to provide some kind of API to other mods?

[raymoo]

Sorry, I mean what if a library is provided as a file a mod includes? The server could send both files and have one docile the other. This will be necessary if you want to share code between the server and client components of a mod.

[BluebirdGreycoat]

The server can send both files and the client can execute both files, order doesn't matter as long as the dependencies are resolved at runtime and not required at loadtime. One file shouldn't specifically need to dofile() another file.

My objection to a virtual filesystem is that it's difficult to verify the implementation, and not strictly needed for CSM to function. If it can be written and verified I have no objection, as long as it's read-only (because otherwise, cookies!).

[raymoo]

Well, you could write the filesystem in sandboxed Lua and just have the core code call the Lua functions to add files. I think it doesn't have to be read-only as long as it's not persistent, but writing to a non-persistent filesystem is probably not that useful.

It doesn't have to implement the full file API either, it could just implement dofile only.

[paramat]

Can anyone help with #6699 "Server setting for complete disable of client-provided CSM"?

[BluebirdGreycoat]

The issue is that any file access requires the Lua VM to expose a read function which runs _outside of the sandbox_. That function will need to take a filepath as a parameter. Ensuring that the given filepath points only to allowed locations on the user's filesystem (or in-memory filesystem) is not at all trivial. There are problems with symbolic links, hard links, recursively linked directories, and other snarly filesystem entities. PhysicsFS is a library that could help, but I don't know much about it.

The CSM API should be as minimalistic as possible to make auditing easier.

[BluebirdGreycoat]

The specific example you gave, I think can be solved as I hinted at already. The server would send the CSM and any library code the CSM needs as multiple individual chunks, and the client would execute each chunk in whatever order (execution order doesn't need to be defined, I think). Then, as the client calls any callbacks that the chunks registered, the CSM can resolve its dependencies at runtime, since, if written correctly, all the library chunks should have exposed their APIs through tables.

Basically (client-side pseudo-code),

local function csm()
    csm_library_api_table.some_api_function(...)
end

client.register_some_callback(csm)

That will always work even if the library is loaded after the CSM itself is loaded.

[BluebirdGreycoat]

The technique I wrote above is one I use frequently. It is especially useful when there are irresolvable circular dependences between mods and depends.txt is useless.

[raymoo]

I was suggesting just having a dofile that loads the script that originally had the specified filename on the server. It doesn't need to do any file access because it would just run a string stored in memory.

[BluebirdGreycoat]

Ah, I see. So the server would give the filename (or any arbitrary string ID) of every chunk sent, without regard for path information (so no two chunks could have the same "name"). Then on the client you'd do dofile("libABC"). But in that case it probably would not be good idea to call the function dofile, it could probably be called require instead (the real require should not be in the sandbox, it is unsafe).

And to be clear, you could only require with string IDs that the server actually gave you. The client should not go looking around on the filesystem for some other library that happens to have the same name.

[raymoo]

The name should depend in the semantics. require will return the old value without running an already-ran script, while docile will run it again.

[rubenwardy]

Here's a quote from Mike Pall, the author of LuaJIT:

Maybe for an extremly restricted subset of Lua and if you
relentlessly scrutinize every single interface function you offer
to the untrusted code.

Others have repeatedly failed to keep their sandboxes sealed. See
Java, see Flash, see Acrobat. And they had 15 years of time to get
it right. Why do you believe you can do better?

http://lua-users.org/lists/lua-l/2011-02/msg01606.html

so even when we put a lot of effort into securing the sandbox, there should still be precautions in place in-case there's a mistake

[paramat]

This is possibly concerning https://github.com/minetest/minetest/issues/6699#issuecomment-347671425 although maybe i misunderstand.

[paramat]

With nerzhul and red-001 inactive, it seems we are a bit stuck at the worst point of CSM dev, before the truly useful stuff is added, but after client-provided CSM has been added which is causing problems for little usefulness.

It's clear that server-provided CSM has to be done very carefully and is a risk, so will need a big commitment from those who code it to test it. It seems unclear whether the CSM devs will have time or still have the commitment. So what do we do in the meantime?

Seems best we completely disable CSM in 0.5.0 stable while server-provided CSM is worked on (if it ever is), until we have that it is not worth having client-provided CSM usable as the problems outweigh the minor benefits. I still haven't seen any CSMod that seems to justify or that could not be done by a server mod. The forum section seems to be lots of messing about and lots of cheat mods that we keep having to delete.

Any support? I suggest we work on that now. I won't be able to do it myself unfortunately but can make the PR if others help.

[BluebirdGreycoat]

Yes, just strip out the ability in the client to actually load CSM (no need to remove the API itself). Then later focus can shift to creating a client-side sandbox, which I'll happily test (though obviously it will need many testers). And much later I suppose focus can move to making it possible for the server to send Lua chunks to the client.

But at no point should this become usable to regular users, and certainly not enabled by default, for a long time.

[paramat]

Yes as we know now, it should never have been allowed in 0.4.16 stable, if we had done the right thing it would still be disabled and in long-term development now, which is exactly what i am suggesting.

[BluebirdGreycoat]

Also, I want to warn against the tendency, while developing the client API, to add functions simply because a dev feels the function is trivial. I propose that the API should be as minimalist as possible. Ideally every function proposed should be talked over on the forum and with server operators and users, so we know what we're getting, and the probability that someone spots something dangerous increases.

Every function should be discussed, even pairs()/ipairs(), if it calls into C/C++ code. (Silly but leave no stone unturned.)

[paramat]

Which is why #6670 should be merged.

[BluebirdGreycoat]

See #6710.

[paramat]

Yes, just strip out the ability in the client to actually load CSM (no need to remove the API itself).

This is actually similar to what i am suggesting here #6699
Oh i see you're there already =)

[paramat]

I think we should have stuck to the original plan http://dev.minetest.net/Client_scripting_plans
Note no mention of client-provided CSM:
////////////
General design

The language will be plaintext lua (no bytecode, as it's insecure and against the open source nature of minetest). There won't be any restrictions on mod licenses though.

The general design should be much like the Web browser-JavaScript relationship. During the connection, the server sends all code that should be executed to the client, which runs it inside a locked down sandbox. Unlike with server mods, there will be no way (without changing C++ source code) to execute code outside of that sandbox.

Unlike with Web browsers, Lua code should run in a separate thread. Also, to be completely separate, it shouldn't share the Lua environment with the mainmenu Lua (to forestall intra-environment exploits).

[paramat]

Any help with sorting out CSM is very much appreciated, i feel we need some fresh eyes and new opinions on the implementation.

[ghost]

It's clear that server-provided CSM has to be done very carefully and is a risk, […]

Before server-provided CSM goes live there should be something done about the insecure connection. Otherwise MITM attacks would allow a malicious person to inject arbitrary code to be run on clients. This could expoit the environment and do actual harm or simply be annoying.

Seems best we completely disable CSM in 0.5.0 stable while server-provided CSM is worked on (if it ever is), until we have that it is not worth having client-provided CSM usable as the problems outweigh the minor benefits.

This and 100 percent of this. Disable this gaping security hole. Work on secure connections, and then, iv anyone is interested coding it, work on server-provided CSM (whjere I still don’t see one single use case for) if you want.

I still haven't seen any CSMod that seems to justify or that could not be done by a server mod. The forum section seems to be lots of messing about and lots of cheat mods that we keep having to delete.

Yes, currently the main purpose of CSM is cheating.

Btw. How old is the client-side scripting roadmap? “_Unlike with Web browsers, Lua code should run in a separate thread._” isn’t true for any of the relevant browsers since a few years.

[Wuzzy2]

First of all, I want to note that the challenge to post some compelling real world use cases for server-sent client mods still remains unanswered.

@Wuzzy2: I think it is clear by now that you actually have not researched sandboxes for yourself. Nor do you appear to have experienced for yourself how much easier they make development.

Jeez. I'm not saying that sandboxes are the work of the devil, nor do I think they are useless. I'm NOT against using sandboxes. I just say that just because we're using a sandbox (which we definitely should, by the way!) does not mean we're in Magical Happy Land now and that there's no way in the world to screw up.

So because it's insecure, it okay to increase the attack surface?

A sandbox does not increase the attack surface. (…)

My quote is taken out of context. “It” refers to Minetest as a whole, not the sandbox.

also find it odd that you seem not to have noticed that I support having server-sent CSM be disabled by default during the testing period. In other words, I support opt-in.

This opt-in will likely become opt-out if server owners start to lock out players, either directly or indirectly (by adding features which only work by accepting scripts).

Let's wait for empirical data before blindly believing that most servers will actively kick players who refuse CSM.

It's a possible scenario and an unneccessary risk. What's certain is some servers will be off-limits if you disable this in your client. What I'm fearing is that it will be a slow process so that it goes by mostly unnoticed until it is too late.
Also, “collecting empirical data” means implementing the whole system. Nice try.
JavaScript in the Web also started as “optional” until more and more websites cease to function without JavaScript, making it de facto mandatory.

Mathematical proof of safety is impossible at this level, especially when "unsafe" languages such as C or C++ are involved. (Not sure what kind of different proof you want.)

Agreed. Port Minetest to Haskell then. XD
Joke aside, with “proof” I meant that it should at least undergo some serious testing and auditing before it is even considered to be released.

Just because this is a FOSS project, doesn't mean Stallman has to be followed by the word.
Continuing the analogy with web browsers: Very few people see a conflict between free software (e.g. Firefox) being used to run propriertary code, and the ones that do are already using LibreJS

This is a terrible reason. This is just a hypocrisy at work here. If you think that the idea of FOSS is true, then is completely nonsensical to single out scripts. By the same logic, there would be no problem in releasing Minetest Game under proprietary terms because it's “just” a scripted language.
Also, I don't get why you insist on the possibility for server operators to send proprietary code.
Why? Do you plan to ship proprietary code on your own? Nobody exept the owners will benefit from this.
What you're essentially saying is that it's perfectly fine for server operators going after their users for copyright infringement because they copied some of their code without permission. Bullshit like this is the whole reason why we do free software stuff.
Free software is essentally not being a dick to your users (by sueing them for copyright infringement). I'm essentially asking developers and server operators to stop being dicks to their users. A free software license is a public statement that yes, you're not a dick to your users.

And how on Earth do you make this system usable and understandable for the average player (including kids)?

You don't. Have you seen children caring about software licensing? Me neither.
Just as with a "NoScript" browser addon, such functionality is for users that would like to have more control over which code they run and that understand the concepts behind CSM and server-provided mods.

Well, that's my point! The average user is exposed to risks he/she does not understand. This is simply irressponsible to ask the user to sort out the security problems by hand.
I don't get why are you even talking about NoScript-like in the first place. Aren't you implicitly admitting that there are inherent problems with the system? If the system is so great then the need to disable it wouldn't arise. We also don't make every Minetest feature optional just because we could.

Also, “people caring about licensing” is missing the point. These people were just lucky enough to never got sued so are.

The complexity explodes because now you have to maintain two Lua APIs. Maintaining just one Lua API is already time-consuming.

Maybe you have missed it, but CSM does already exist and with it the "two Lua APIs" you claim to be so disastrous for everything that ever touches it.

Frankly, the client mods I've seen so far don't impress me. And most of them are used for blatant cheating. It should also be noted that CSM is not really popular among modders.

I find it hard to take someone serious [...]

A cheap and inaccurate shot, that was.

I noticed you still use the word “internet” to refer to the WWW. Come on! Seriously?

Well, since people with that philosophy (with the exception of RMS) typically do work on the internet anyway, thus allowing proprietary code to run in their browser, I honestly don't take the philosophy seriously. This is something that kids get right by not caring. Whether some software has the GNU badge or the FOSS badge or the OSS badge or the M$ badge doesn't matter. People who write software will either treat your data as private or they won't, and what license they choose does not restrict their choice.

Nobody wants to deal with copyright BS in the Internet. That's only natural. Until you get sued for copyright infringement. That's the whole reason why we even do free software stuff! As long there is copyright, there will always be those annoying people who insist on FOSS. You know, about not being a dick to your users. ;)

The amount of code will explode.

Nothing wrong with that, as long as that code isn't in the engine itself. In any case this situation already exists, just look at all the server-side mods we have.

Mods will inevitably contain a lot of bloat and implement the same feature over and over again.

Uh, what? First I doubt this will be the case, and if it was the case then we'd have that problem already. How many protection mods do we have, currently? And yet none of them are exactly the same. Variety is good.

Yeah, right, beause duplicating the same feature over and over again is a such great idea. I'm looking at you, mob mods! It's so fun in having to deal with dozens of slightly similar yet completely incompatible mod APIs. And it's even more fun when mods start to depend on all these APIs. Have fun compiling a set of mobs out of these mobs in order to make a subgame.
I challenge you to try it and then tell me again there's absolutely no problem with code duplication.

This is the kind of crap I have to deal with as a modder and subgame author. My fear is that it will only get worse with this feature.

I don't like minimalist APIs which lack even basic functionality so scripts are forced to re-invent the wheel.

The smaller the API, the better for security and verification. That being said, there is nothing wrong with having mods that overlap or even duplicate functionality. Variety in implementation is a good thing.

I tend to disagree. It is nonsensical to have dozens of slightly different mod APIs for the same thing. Also, I know it can be seducing to have a simple and generic but very powerful engine, but it's easy to fall into the trap of completely forgetting the actual goal of all this, namely creating actual games. This should always be kept in mind when designing the API.

Solving a problem with hand-written C++ code in the engine is not less code, and is not safer code, than just running a script in a sandbox. Also, bugs in server-sent scripts are a problem for server operators, not the Minetest devs. And we already have a shared common base, that is, builtin and minetest_game to a lesser extent.

I'm perfectly fine in putting stuff into builtin, however.

Enabling server-sent code in Minetest effectively allows server operators to extend the network protocol without having to wait for the devs.

I fear that this will become just yet another excuse to not add core features to Minetest because the task has been outsourced to modders.

The reason there is a modding API in the first place is to avoid bloating core with a bunch of features where each feature isn't needed by that many people. An example is GUI / HUD widgets.

This example is horryfying. Now we modders shouldn't even get the freaking widgets for free? This stuff is essential, there is no reason to require modders to reinvent buttons and stuff. I don't want to re-invent the wheel when creating a subgame! This line of thinking is exactly why I am strongly against a minimalist engine.

Should every desired widget be implemented in core?

No. But there's a lack of important widgets, such as scrollable multi-line text.

---

@BluebirdGreycoat: Who are you, anyway? You seem to have appeared completely out of nowhere in Minetest yet you speak as if you have been involved in the project for many years. It's very strange. Did you ever write a mod for Minetest? Which ones?

[BluebirdGreycoat]

@Wuzzy2, I'll reply only to the points of your post that specifically refer to me mainly because most of the rest of it has either already been answered, or is based on belief/concern more than data.

I noticed you still use the word “internet” to refer to the WWW. Come on! Seriously?

If you imagine that the WWW is the only place where the discussed technologies are used, you are mistaken. My remarks apply to the internet as a whole, as far as security (or the lack thereof) goes. For that reason my use of the word is correct. Even when it is not correct according to your sterile, formal rules, it does not affect my arguments in the slightest. Frankly, to even remark on it is slightly elitist. Will you complain about my missing commas too?

You seem to have appeared completely out of nowhere in Minetest yet you speak as if you have been involved in the project for many years.

No, I merely speak with confidence that I understand the subject discussed. I think you're projecting, honestly.

It's very strange.

It's strange for someone with a stake in the issue (as both a user and a server-op) to speak up when they see that something has gone wrong?

Did you ever write a mod for Minetest? Which ones?

For the Minetest community? No. For myself and my server? An entire game.

[raymoo]

@Wuzzy2 I never said modders would have to write all the widgets, just that client CSM would be necessary to sanely handle custom widgets. I agree that a suite of generally-useful (and maybe some that are specific to popular types of games) should be provided.

I can't remember if I said this already in this thread, but I wouldn't mind too much having to download CSMs for a mod separately (not automatically sent from the server). It's less convenient, but clearly it is workable since people have to deal with it in Minecraft. The problem in Minetest would be that there is no way to manage different versions of the same mod, since different servers may use different versions. It would also make it harder to prevent clients from providing their own client mods, since they could just modify the ones corresponding to server mod. This might be solved, at least for unmodified clients, by hashing each client mod and sending it to the server for verification. Like mentioned in some other issue, someone could modify their client to send fake hashes, but then they would need to modify their client or get it from a third party that they may not trust.

[paramat]

I can't remember if I said this already in this thread, but I wouldn't mind too much having to download CSMs for a mod separately

See nore's comment http://irc.minetest.net/minetest-dev/2017-11-29#i_5151215

07:00 nore IMHO, concerning #6662, we should give the choice to the user between downloading and installing mods manually (with a hash check to ensure the same mod runs on the server), or automatically from the server while the mod is signed by its author, giving the possibility to the user to trust some mod authors (and needing to individually accept mods that are not by these authors or are not signed)
07:01 nore The rationale behind all that is that whatever sandbox we do, there will be a way, at some point, to escape it -- sandboxes are buggy
07:03 nore On the other hand, downloading mods separately by the client already implies trust in the given code - and malicious code can be far more easily detected and removed
07:03 nore (As with server-side mods)

[paramat]

Seems best we completely disable CSM in 0.5.0 stable while server-provided CSM is worked on (if it ever is), until we have that it is not worth having client-provided CSM usable as the problems outweigh the minor benefits.

I still think this is best, but this has been disapproved by 3 core devs. However i think it is reasonable to give server owners the 'master disable' option, which was my original intention of #6699 This has the advantage that is is then easy for a server to participate in testing and improving CSM, just set the setting. Also, since CSM issues affect servers it is reasonable that they have the choice.
If i can i'll code it, any help appreciated.

////////////

I feel i have to make something clear: the strength of feeling over CSM and the seriousmess of the situation. Most of our most respected server owners are extremely concerned and some are considering radical actions. As a multiplayer game engine, without servers MT is nothing. We have to listen to server owners and make some concessions and compromises. CSM can go ahead, including server-provided CSM, but with more care, more testing, more time and with more optional restrictions. Importantly we have to listen to server owners more and involve them more.

[Wuzzy2]

I can't remember if I said this already in this thread, but I wouldn't mind too much having to download CSMs for a mod separately

This is a terrible idea!
If players will be forced to manually download anything extra in order to play on a particular server, that would be the death of Minetest.
It's a major strength of Minetest that is just works out of the box; you can just join any random server, Minetest does the rest for you.

Requiring to download anything extra by hand means we lose one of the most important core features of Minetest and also one of the advantages over Minecraft. I don't have words for how unbelievably stupid this idea is.

If (IF!) you introduce server-sent client scripts, please at least make sure that everything happens automagically, like now. Anything else will be a major regression.

---

EDIT:
Concerning trust features, I think that everything should happen in the application itself. I think that nore's proposed NoScript-model for Minetest is flawed because it requires user intervention. Also, malicious code is malicious, no matter its origin. Most players will be too stupid to use these features, they don't care about this trust model and just trust everyone to get rid of the warning. Which is probably not the smartest move. You seem to forget that most players are NOT hackers like you. So I think the security model needs to be rethought. This NoScript-like model is not convincing.

I 100% agree the sandbox should be really minimalist, especially at the beginning. Less chance to screw up. And then work from there, but each new added feature needs careful consideration.
If the possible client code can only do this-and-that, I don't really see the need for a full-blown NoScript thingie.

My biggest concern is still that the implementation of the sandbox itself in Minetest will screw up. NoScript won't help you there. The implementation of the sandbox should definitely not be rushed in.

[sofar]

Requiring to download anything extra by hand means we lose one of the most important core features of Minetest and also one of the advantages over Minecraft. I don't have words for how unbelievably stupid this idea is.

This is for me the bottom line as well. I've had about half a dozen ideas already implemented and all coded up if it wasn't for the fact that I do not ever want these bits manually downloaded by players (ever).

I am concerned about sandbox escapes, but I think we as a community have a big stick in hand by controlling the master server list, and it will absolutely allow us to completely banish malicious server operators - that send bad code to players - from the community. This alone is probably a much more powerful tool than anything else.

We also have lots of tools in place to collectively assure servers on the server list behave according to certain standards, even before players would be able to see them on the actual game client. We can absolutely vet them and monitor them.

[sofar]

@BluebirdGreycoat

Some of us are forced to disable anticheat on our servers. The fast-move prevention in particular is a royal pain on a laggy home connection. And on "Inside the Box" for example, (I believe that server has anticheat enabled) I get yanked around quite a bit as soon as I fly around in edit mode for more than 5 seconds straight. This is due to anticheat's fast-move prevention AFAIK. Count me among those very upset server owners.

That's my server. Anticheat isn't ideal, but your connection is ultimately the cause. Feel free to submit issues to anticheat, though, or provide a better implementation? I would certainly appreciate improvements.

For reference, anticheat is enabled by default in minetest. It's not something server operators enable. Some may have disabled it, but it really should be enabled by default, even despite its flaws.

[Wuzzy2]

I am concerned about sandbox escapes, but I think we as a community have a big stick in hand by controlling the master server list, and it will absolutely allow us to completely banish malicious server operators - that send bad code to players - from the community. This alone is probably a much more powerful tool than anything else.

This kind of power can always be abused (if it isn't already) to censor servers the admin(s) doesn't like personally. It is incorrect to say that “the community” gets to say which and which servers get censored. The admin(s) has de facto the final say, there's no way around that.
Also, this model is flawed. Since if you spot a “malicious server”, it's already too late, the damage has been done. That's not convincing.
The focus should really be on making the code more robust in the first place.

[sfan5]

(if it isn't already)

lol'd

[ghost]

This kind of power can always be abused (if it isn't already) to censor servers the admin(s) doesn't like personally. It is incorrect to say that “the community” gets to say which and which servers get censored. The admin(s) has de facto the final say […]

While this is a 100 percent correct statement people can always set serverlist_url to another master server. Not sure if this affects server announcement as well. But the master server code is available.

If it’s true that servers are rated worse or sometimes not listed due to serverlist admins modifying the results then it wouldn’t take long until the community gets split into “I don’t care.” and “We don’t like being ‘censored’!” sunning an own instance of the master server.

But you’re right, this has nothing to do with the insecure CSM that should be disabled and reworked completely (if not simply removed).

[paramat]

I should warn that the situation behind the scenes is worse than you realise.

Also, it seems likely from public statements that RedCat creative and DarkLands, 2 of our top serves, will become whitelisted whitelist players. Not entirely due to CSM of course, but that is the issue that adds to others to cause this.

We devs have to realise the seriousness of this and compromise a bit, not get carried away by the exciting possibilities of CSM and be practical, CSM has to rolled back a little in some form, this doesn't mean it had to be removed completely, but removing client-provided CSM once server-provided CSM has been added should be considered, since that was the original and long-agreed plan.

Considering that almost all of our top servers are extremely concerned about CSM, something has to be done. Please can certain devs not be so stubborn for the sake of MT? Without servers MT is nothing.

@celeron55 any comments, as our 'conflict resolver'? =)

[sofar]

2 of our top serves, will become whitelisted

What does this mean? Who whitelists what?

[BluebirdGreycoat]

It's fairly common for server operators to allow players to join and make new accounts without requiring them to jump through hoops or sign up for stuff in their browser (or even pay money). This is indeed a great strength of Minetest.

Whitelisting is the implementation of some system designed to either prevent or make it difficult for new players to join the server and play with the same opportunities as every else. Whitelisting is implemented at the discretion of the server operator. The implications, if Minetest's most popular servers adopt such a system (in a bid to prevent players from gaining unjustly acquired technological advantage over other players), are not something I'm qualified to speculate about.

The strikeout is because there are actually a number of valid reasons for servers to implement such a system that are related, in part, to the current implementation of CSM, which are not limited to cheating. I don't mean to exclude those reasons from this post.

[sofar]

Ah, no wonder I didn't understood this sentence.

that ... 2 of our top serves[sic], will become whitelisted

The servers are not getting whitelisted by someone or something, paramat should have said somethng like

These servers will start whitelisting players

or something unambiguous like that instead.

[sofar]

Heck even saying "servers .. will become whitelisting" would make it clear.

[sofar]

But, is that what paramat actually meant ?

[paramat]

Here's what celeron55 wrote in IRC http://irc.minetest.net/minetest-dev/2017-12-03#i_5154905
I agree that server-provided CSM needs to be added to justify CSM.
////////////

07:04 celeron55 it's a lot of backlash for sure
07:04 celeron55 also, there's barely any middle ground that makes sense
07:04 celeron55 to be taken, that is
07:05 celeron55 i mean seriously, it's a real possibility that we just need to rip out CSM and live without it
07:06 celeron55 it's useless to try to be idealistic about it
07:10 celeron55 but i think people are misjudging the effect of CSM; cheating has been on the rise anyway
07:10 celeron55 it's a really bad time to be fueling cheating with CSM though
07:14 sofar can't ignore the cheating by waving away csm
07:14 sofar we need some serious action on that side
07:14 celeron55 of course not, but you can't fix it in a short timespan at all
07:14 sofar and if people who run servers aren't jumping in, they are just as guilty as cheaters
07:15 sofar they're effectively enabling it
07:15 sofar I agree there should be a compromise btw
07:15 VanessaE I'd jump in but my typing is compromised. I can't keep up.
07:15 celeron55 the security issues pointed out by Wuzzy are accurate, personally those have been my main concern ever since i decided client-side mods downloaded from the server could be interesting; and given the lack of trust to the programming skills of anyone... ehm
07:15 sofar but it shouldn't be "remove csm"
07:16 sofar VanessaE: just say "warm", "colder", "freezing"
07:16 VanessaE heh
07:16 celeron55 for one, i think leaving csm in without code downloaded from the server is stupid
07:16 sofar agreed
07:16 celeron55 either that or nothing is the future
07:17 sofar I've said something similar a few times
07:17 celeron55 i'm fine with either
07:17 VanessaE my opinion is that we should have never had client-provided csm. it should have been server-provided from the start
07:17 sofar I will start making CSM mods when servers can send code
07:17 sofar not before
07:17 sofar well it's a good lesson
07:17 sofar I don't think nerzhul and red thought it through either
07:18 celeron55 VanessaE: that was my plan, but things tend to go the way the actual implementer does it
07:19 VanessaE I know. just saying.
07:19 celeron55 it's very much a "the kettle fell on the floor and now your feet are in boiling porridge" situation
07:19 celeron55 it doesn't help telling yourself you shouldn't have done it
07:20 celeron55 oops, as the linux kernel likes to say
07:21 celeron55 but, like
07:21 celeron55 the complete lack of actual lists of problems is amazing
07:22 celeron55 i can see two: scanning for ores and taking items from chests without collision detection
07:22 VanessaE shouldn't the latter be entirely the server's job?
07:23 celeron55 in the design MT happens to be using both of those could be made impossible by the server; the first one takes more effort and server resources to implement though
07:23 celeron55 the second one was probably assumed to be already checked but apparently isn't
07:24 celeron55 the first one could be made impossible by not sending ores that aren't close to the player and next to an open space
07:25 celeron55 needs constant updates when moving though; it's not elegant, fast or cheap at all, i did that in a 2d tile-based game
07:26 celeron55 as all CSM hacks are doable in C++, even if CSM was completely removed we still have to fix all of these
07:26 celeron55 ALL of them
07:27 celeron55 there's no way around it and doing anything to CSM doesn't help
07:29 celeron55 that's the logic due to which CSM was deemed fine to begin with
07:29 celeron55 it's an open source client, how much more open can you make the source? In theory, not at all; in practice, apparently somewhat
07:29 sofar one of the biggest problems is insecure server mods with gaping holes
07:30 sofar minehacker caused us to fix the most egregorious core holes already
07:30 sofar csm SHOULD cause us to hunt down bad server mods
07:30 sofar but it isn't
07:30 celeron55 again not in theory related to CSM, in practice CSM makes everyone a greyhat hacker
07:31 celeron55 +but
07:31 sofar sure
07:33 celeron55 it's kind of cool from every perspective other than running a non-whitelisted public server
07:33 celeron55 ...which MT historically has been able to do, against all odds
07:34 celeron55 literally i don't even understand how that has been possible in practice, but that might be history no matter what is done
07:36 sofar what is this server whitelisting?
07:37 celeron55 i mean, whitelisting players by servers
07:40 celeron55 the thing that every online game does at least in the form of having to register and activate an email for the player account but usually by paying the game company or by being friends by the server host
07:40 celeron55 friends with*
07:40 sofar IC
07:43 celeron55 some people seem to come from the perspective of never having done any of that, but it is a widely known and widely used solution to the universal problem of cheating
07:46 celeron55 the more tedious, expensive or friend-based you make it to get to a server, the smaller ratio of cheaters there will be

[paramat]

Yes i mean the servers will whitelist players, no public player access.

So concerning the necessity of adding server-provided CSM @red-001 @nerzhul are you able to work on this soonish? Although nerzhul i now you are busy currently. Red-001 told me there's not much more to do, we would just need to test it very carefully over a long time (it may not be ready for 0.5.0, which is fine). However red-001 isn't responding to tags or on IRC, which concerns me.

[ghost]

To me the most important line in this chat log are at the very top.

07:06 celeron55 it's useless to try to be idealistic about it

Some tend to idealize CSM (or server-sent CSM when it comes to server owners) as the holy grail every modder, player and server owner wants and that will solve everything entirely.

But that isn’t. In the current state it’s a tool for make cheating very easy and server owners are powerless against it. There are no real use cases for server-sent CSMs that could’t be better solved in core either server-side or client-side.

If server-sent CSM will ever be implemented without making the client-server connection secure (TLS) anyone can inject arbitrary code that will be executed on client-side.

In my opinion the first step should be disabling the whole CSM system intirely in 0.5.0. Then reconsider the need of CSM: actual use cases, extensive security improvements (connection security, sandboxing, etc.), configurability for clients and vor servers (disable entirely and do not crash if disabled, configurable white-list of server-sent mods for individual servers, etc.).

[Ezhh]

As a server owner, I'm mostly indifferent to server-sent CSM, though my level of faith in it being implemented well (especially in any timely manner), is very low.

If (massive big if here) it can be done well, okay. But since we've not seen any public response from those who worked on CSM so far, I'm completely in favour of CSM being removed. (To those who want to insist server owners don't understand: this is said with full awareness that this is not a way to guarantee no cheating will ever happen again).

[paramat]

@nerzhul any idea what happened to red-001? I keep tagging him and highlighting him on IRC but no response. Apparently he has server-sent CSM almost ready, perhaps you could continue his work yourself?
He also has a large number of PRs that are neglected, he can't leave CSM half done.
Essentially as you can see, it's either server-provided CSM is added or all CSM is removed, so it would be good to know stuff is going ahead.

IRC:

07:16 celeron55 for one, i think leaving csm in without code downloaded from the server is stupid
07:16 celeron55 either that or nothing is the future
07:17 celeron55 i'm fine with either

[nerzhul]

i don't know what happen, but like ourselves red-001 is not paid and is just a contributor and can disappear. Many CSM PR are not merged due to other coredevs not reviewing them blocking some interesting abilities :(

[ghost]

celeron55 basically saying "either server-provided CSM or nothing".

Since one of the two active CSM devs disappeared and the other devs are not able (no time, not into the project, etc.) to do something and most if not all server owners are against it I'd say bury CSM once and for good.

[sofar]

@dsohler As long as there is one active developer working on a feature set, it's not abandoned.

[neoascetic]

leaving csm in without code downloaded from the server is stupid

That what we meant CSM must be when created the issue in the first place:

the scripts that should be sent to clients like media files

[paramat]

6738 "CSM fixes: load mods after flavours & add flavour to block mod loading" has been merged, this adds the ability for the server to 'master disable' all client mod loading using CSM flavours. It has some extra work to be done, making builtin secure from alteration.

Also it seems nerzhul may be workng on adding server-provided CSM soon =)

[ghost]

Also it seems nerzhul may be workng on adding server-provided CSM soon =)

Damn … :rofl:

[Wuzzy2]

… and I'm still waiting to hear actual use cases for this.

[neoascetic]

Weather mod to spawn particles locally and not on a server

On 13 Dec 2017, at 04:29, Wuzzy [e-mail] wrote:

… and I'm still waiting to hear actual use cases for this.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[ghost]

Weather mod to spawn particles locally and not on a server

That's the job of a weather API.

[neoascetic]

-- I need CSM for FOO
-- That's the job of a FOO API

As a (bad) analogy is Web1.0 vs Web2.0

[ghost]

You can't compare a web browser and a voxel game engine.

[neoascetic]

I just did it.

I mean, I could compare them as client-server applications. I would prefer to have a rich client. See, CSM is API as well. But since it provides an ability to execute some programming language is more dynamic than the static set of API endpoints. Of course, we have the programming language on the server. That's the question of what functions better to shift to client-side. I am sure that more dynamic environment and data-locality leads to the better user experience.

On 13 Dec 2017, at 12:48, Dirk [e-mail] wrote:

You can't compare a web browser and a voxel game engine.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[Wuzzy2]

Yes, “spawning particles locally” is an use case, but doesn't sound like it really needs client-side scripts. But I think we already have “local” particles to some extent. We have particlespawners which already help to reduce the server load (you don't need to spawn each single particle by hand). There's another problem: The weather state itself will probably always be on server side anyway. So the server has to send some packets anyway until the clients see the weather change.
So, this use case doesn't sound convincing, it just increases mod complexity.

To convince me, the use case has to be really compelling. Something that would be a major improvement for which there are no good alternative solutions. Not just a “nice to have” feature. Remember, server-sent scripts are a GIGANTIC change in how Minetest mods work. It's almost as significant as the introduction of modding support itself. Adding such a change to Minetest should have a really good justification, and not just because you feel like it. Note I'm a long-time modder myself.

I think a lot of use cases could be implemented simply by improving client prediction and/or the net protocol. I don't see anything wrong with this. It reduces complexity of mods considerably, and ALL subgames will equally benefit, not just a few.

---

Anyway:
If you STILL insist on server-sent mods:

Do we even have people who have the neccessary skill level to implement this securely?
Who will verify the code?
And how do you verify that you didn't screw up the sandbox implementation or anything else?
Also, PLEASE don't rush this in before 0.5.0?
The track record with CSM so far is not convincing.

Also, I dislike that my licensing concerns have always been dismissed as off-topic.
Sorry, but I don't think it's worth opening a new issue over this, however.
Frankly, licensing should be discussed NOW. If we discuss it after-the-fact, it will
be nearly impossible to change later, and client computers will be quickly
polluted with non-free software, defeating the whole point of Minetest.

[raymoo]

I agree with the free software concern. It should be possible for a user to reject non-free software from the server.

As a side note, I wrote #6688 which is supposed to make particle spawners a little bit more flexible. Though, CSM with some kind of batch spawn particle (spawning one particle at a time will have lots of C<->Lua FFI overhead) would be the most flexibility possible.

[sofar]

Yes, “spawning particles locally” is an use case, but doesn't sound like it really needs client-side scripts

uhh

Weather mod to spawn particles locally and not on a server

That's the job of a weather API.

This is a misconception, I don't think both of you understand things correctly.

There's a major difference between what the engine can do, and what the server can do.

Sure, the server could send all the individual weather particles to every connected client. This would quickly fill up the packet queues, unlesss you prefer homeopathic weather particles, because those would work fine. But a realistic rain shower with a few hundred particles is going to max out the max_send_queue and completely break the game experience.

The client can handle thousands of particles just fine, as demonstrated in one of my client side particle tests. It could do this entirely without server-side intervention. All it would need is a single packet to the client that says "it's raining" or similar, and then in CSM generate the correct particles.

In short: the client could, through CSM, do a lot of things that are impossible right now, due to the sheer limitations of the physical reality of a server-client architecture.

[sofar]

Also, I dislike that my licensing concerns have always been dismissed as off-topic.
Sorry, but I don't think it's worth opening a new issue over this, however.
Frankly, licensing should be discussed NOW. If we discuss it after-the-fact, it will
be nearly impossible to change later, and client computers will be quickly
polluted with non-free software, defeating the whole point of Minetest.

Open a separate issue for that, so it can get a fair discussion?

[neoascetic]

I meant that previously it was a bad idea to track user position to spawn particle spawners for rain. That worked bad. I just forgot that we have attachable particle spawners now, but that’s just an example.

On 14 Dec 2017, at 08:40, sofar [e-mail] wrote:

Also, I dislike that my licensing concerns have always been dismissed as off-topic.
Sorry, but I don't think it's worth opening a new issue over this, however.
Frankly, licensing should be discussed NOW. If we discuss it after-the-fact, it will
be nearly impossible to change later, and client computers will be quickly
polluted with non-free software, defeating the whole point of Minetest.

Open a separate issue for that, so it can get a fair discussion?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[rubenwardy]

I think, as a general-rule, that client-side modding should only be for latency sensitive usecases. This includes client-side prediction, and audiovisual effects. I'm in favour of removing CSM if it isn't mainly targeted at the those cases. I, however, think it's recoverable.

The two primary use cases are:

• GUI/HUD - drawing of custom elements similar to HTML5 canvas. Good for computer mods, advanced inventories, and things we haven't thought about yet.
• Entities - ability to locally control entities and perform client-side prediction on them. This will allow carts without any lag, and other types of entities such as Advanced trains. This need to be done right, however.

Due to the mess which is the current CSM implementation, neither of these are currently possible.

There are more usecases, such as environmental sounds, but I argue that they should be part of the C++ engine by default anyway.

[Wuzzy2]

Great!
The issue has been closed, so I conclude that ALL security issues caused by CSM are fixed. Am I right? :wink:

[SmallJoker]

The issue has been closed, so I conclude that ALL security issues caused by CSM are fixed. Am I right?

Partially. This issue exceeded the scope anyway as it covers multiple issues with CSM:

• [x] running bot-chat-text _-> ???.. we have chat limits now_
• [x] opening ANY chest
• [x] to steal items _-> fixed in MT, the rest are mod issues_
• [ ] finding ores without the need for digging _-> can't do anything against modified clients_

[paramat]

In a way not closable yet, as builtin still needs securing, but there is another issue for that #6925 so that could cover that. Also of course, the security issues of CSM are far from over, server-sent CSM raises more issues. But i'm not asking for a reopen.

[beyondlimits]

Haven't read whole issue. CSM is a great idea and I only hope the developers did not break anything because someone came up with a broken idea.

This is my computer and I will decide how it works.

@tenplus1: You only have control over the server. Keep it secure then. Don't volunteer data the clients don't need and don't bother developers to break CSM.

[paramat]

This is my computer and I will decide how it works.

That's invalid, the issue is that CSM can sometimes affect the server and disrupt the server, affecting other players and their enjoyment, in which case it's not a case of your computer working the way you want it too.
If you want to mess around freely you always have singleplayer, but on a server other people can be affected.

No CSM restrictions have been added for aspects that do not affect the server or other players.

You only have control over the server. Keep it secure then. Don't volunteer data the clients don't need

That's a misunderstanding of how multiplayer works and is all invalid. In the case of detecting the location of valuable nodes your suggestion means the server should not send map data to your client, which means your world would have big holes in it, you wouldn't like that. Map data is obviously needed by the client. Keeping servers secure is exactly what we are doing here by making it more difficult to cheat.

[beyondlimits]

Back when I played Minecraft, on some servers there was an anti-cheat module that efficiently prevented players from searching ores by non-legitimate ways (like X-ray). The map chunk was sent to the client, but every ore occluded from all 6 sides was sent as stone. When an user dug one of the neighboring blocks, server sent a packet to change previously hidden stone into ore. Many servers however had a frustrating version of this mod, where random stones appeared as false ores.

EDIT: I've already wrote that on #6114. Find my post and have a good reading.

EDIT: Exactly https://github.com/minetest/minetest/issues/6114#issuecomment-320787774

[paramat]

The map chunk was sent to the client, but every ore occluded from all 6 sides was sent as stone. When an user dug one of the neighboring blocks, server sent a packet to change previously hidden stone into ore.

That's a repulsive idea.

[beyondlimits]

That's a repulsive idea.

Well, so... do you have a better solution?

[rubenwardy]

It's a pretty good idea, but would break node dig prediction - the node is dug before the server says it's ok, in order to reduce the appearance of lag

[beyondlimits]

Well, hmm... you gave me a good point.

[raymoo]

It would mostly work, but still allow finding ores on cave walls (unless expensive line of sight checks are also done).

[beyondlimits]

It would mostly work, but still allow finding ores on cave walls (unless expensive line of sight checks are also done).

Instead of line of sight we could use light == 0, since caves are mostly dark.

[raymoo]

@beyondlimits Then people wouldn't be able to find ores in dark caves.

[beyondlimits]

@raymoo: ~ugh, true... though I laughed considering how dark Minetest was in the past.~

Then people wouldn't be able to find ores in dark caves.

EDIT: Wait a moment. Are players supposed to see anything when light == 0? If the caves are dark, players generally put torches in them, which would cause increment of light parameter and thus uncovering the ores.

[raymoo]

@beyondlimits
It depends on brightness settings. It's usually pretty hard for me to see ores in unlit areas, but I've heard it's different for Windows players. In any case, it's not pitch black.

[paramat]

Well, so... do you have a better solution?

Yes, we have already solved the issue with CSM restrictions and disallowing transparent textures for stone. So there's not much point in discussing bizarre and tasteless solutions. If a player is in a completely dark cave and randomly blindly digs they should be able to dig ores.

[beyondlimits]

@paramat: Guess, I prefer solutions that aren't rendered useless on hacked clients.

Chances that there was misunderstanding in the idea I explained?

[rubenwardy]

If we work to reduce networking lag and improve mesh generation performance, it could be a lot more viable and something that we should support. However, I'd like to see work be done on optimisations first

[paramat]

beyondlimits, good point, sorry i misunderstood.

There's a lot of focus on ores in these discussions, it's not just about ores and it's not just about 'valuable nodes'. It's about being able to detect any nodes that are out of player view, for example hidden passages etc. Doing this only for ores is ignorant and of limited usefulness.

To do this properly you would have to only send the nodes that are visible or accessible to the player and no others, and keep updating this on every node dig, which ends up ridiculously overcomplex and impractical in the short to medium term. However, this is theoretically a good idea given sufficiently advanced technology.

It would also mean only very nearby world is loaded, so when you dig a node network lag will delay the appearence of the node behind. World loading would be laggy. This is the advantage of loading a large radius around the player as we do now.