Multimc5: [Idea] Adding Download Option for Curse Modpacks

[panslothda]

To use a Curse Modpack currently one needs to use something like https://github.com/Vazkii/CMPDL/ or https://github.com/portablejim/curseDownloader And then manually put the downloaded stuff into the instance folder that needs to be done manually and forge installed. Then copy the override folder over everything in the end.

Now i know this isn't really an issue and just takes like 5 minutes every new update for a modpack. But it would be a great enhancement to add something like it to MultiMC if it where to do it all automatically from making instance to download the mods and finishing it all up in the instance folder.

What would you guys think about such a feature? Do-able? or more to much work for not enough gain?

[Multipixelone]

I would love this feature!!! I don't know if it would be do-able or not however, as I only code in python 😝

[neclimdul]

This was sort of rejected here #1649. That said it would be nice _really_ nice since there isn't a native linux client for multimc to do this Like @panslothda said, involved process taking several minutes involving a browser, a terminal, managing a script and remembering all its arguments, and multimc. Pretty annoying.

Even if it was only doing something like dropping the pack url in multimc and it downloading the pack file and ran portable jim's script in such a way that i was in the right place for MultiMC it would be a huge win I think.

[panslothda]

Ye i wouldn't wish for more then a simple one shot import function. Like download the jars of curse put into the multiMC generell mod folder if not there already and then import the ones that the manifest says so into the instance mod folder. this way there would even be caching kinda.

And if it wont happen well i mean its just a bit of time it takes. it would be an awesome feature but by no means anything important.

[StarkRG]

If it could also handle modpack updates that'd put it over and above the current download methods for linux.

[LostOfThought]

It would be in strict conflict with the Curse (now Twitch) TOS.
If we do ever see something of such a sort, I highly doubt it would be part of MultiMC.

[peterix]

I assume that if/when I add something like this, it will be fairly high maintenance... and will be broken sooner or later.

There are other platforms, and much more open.

[target-san]

@LostOfThought Just so I understand things clear. Does this mean no support at all (because Curse/Twitch are such hogs), or just no autoupdates, but a possible import (DL once manually from URL) option?
Thanks

[maxnordlund]

It would be in strict conflict with the Curse (now Twitch) TOS.

I actually read through the TOS just now and it seems like there's no problem with downloading modpacks (or other projects on curseforge) as long as the modpack/mods licenses doesn't prohibit that.

It's only concerned with the rights that you grant Curse, of which there are many:

In addition, by submitting, posting or displaying User Submissions on, to, or through the Service, you grant users of the Service a non-exclusive license to access your User Submissions through the Service (or any successor service) and to use such User Submissions in accordance with the license chosen by you and posted on the Service where such User Submissions are accessible.

Of course I'm no lawyer, and I could be totally wrong, but I don't think downloading packs from curse is a problem. They do have a clause about abuse, like DDOS, so don't download too much.

Would it be possible to depend upon https://github.com/Vazkii/CMPDL and just run it internally? Then the maintenance burden would be significantly reduced. It's written in Java, which shouldn't cause a problem I hope.

[peterix]

2.
...
BY DOWNLOADING, INSTALLING, OR OTHERWISE USING THE TWITCH SERVICES, YOU REPRESENT THAT YOU ARE AT LEAST 13 YEARS OF AGE AND HAVE NOT BEEN PREVIOUSLY REMOVED FROM OR PROHIBITED FROM RECEIVING THE TWITCH SERVICES.

I cannot guarantee that. Technically, any downloader should ask you about your age first. No kids under 13 allowed to download Minecraft mods...

7.
...
This license is subject to these Terms of Service and does not permit you to engage in any of the following: ...
(c) modifying or otherwise making any derivative uses of the Twitch Services or the Materials, or any portion of them;

I will not keep the packs as is, they have to be transformed into a sensible format first.

(d) use of any data mining, robots or similar data gathering or extraction methods;

Required for finding out what is on the platform. Without this or directly connecting to the API (which is private, undocumented and without any guarantees), you have to get the packs manually from the website.

(e) downloading (except page caching) of any portion of the Twitch Services, the Materials, or any information contained in them, except as expressly permitted on the Twitch Services;

What IS actually expressly permitted?

(f) any use of the Twitch Services or the Materials except for their intended purposes.

This could spawn a very long and entirely subjective argument about what is the intended purpose and whether a third party client is or is not violating this undefined intended purpose. Technically, taking the twitch website and piping it to your soundcard's output is a violation of the TOS... and it sounds like binary garbage... and no one cares if you do it or not.

As for the tools, yes. They exist. Writing the code is not the issue either. It is whether I ignore all this or not... and if I will be ignored back. Which would be the best outcome, but not the only possible one.

... and writing this makes me a bit sad.

[maxnordlund]

But those are the Twitch TOS, right? I was reading the Curse one, which is the one governing the Submitted Projects.

I guess down the road that might change, but the steaming service is quite different from the mods and one would hope they recognized that. The _do_ separate the "normal" user content, e.g. forum post, artwork, photos, video, voice, … from the projects on CurseForge.

But in the end it will always be a gamble I suppose. What if the user pastes the download link directly to the zip file? Then you would be scraping the site for what to download as it's all in the manifest provided by the user themselves. While slightly more of a hassle, it's more then good enough for at least me.

… and it sounds like binary garbage :P

❤️

[peterix]

I cannot find a link to that on their website, the TOS link goes to the Twitch one... But while we're at it...

Restrictions on Use
You may not connect to or use the Service in any way not expressly permitted by these Terms of Service. You agree not to access, or attempt to access, the Service by any means other than through the user interface provided through the Service. Without limiting the foregoing, you agree that you will not...

Well, there goes that, now we're off the rails already.

(iv) copy or distribute any part of the Service in any medium without Curse's prior written authorization;
(v) alter or modify any part of the Service other than as may be reasonably necessary to use the Service for its intended purpose;

What is a 'part of the Service'? This probably doesn't apply though. Probably.

(vi) intercept, examine or otherwise observe any proprietary communications protocol used by the Service, whether through the use of a network analyzer, packet sniffer or other device;

Great. Anyone who made those download tools had to do this to some extent. Some more, some less...

(viii) frame or utilize framing techniques to enclose the Service or any portion thereof without our prior written consent (which consent may be withheld in our sole discretion);

'frame' is undefined.

(ix) circumvent, disable or otherwise interfere with security related features of the Service or features that prevent or restrict use or copying of any Content or enforce limitations on use of the Service or the Content therein;

Sounds like this would apply to impersonating a browser while downloading mods/packs. Also, DMCA.
See:
https://github.com/Vazkii/CMPDL/commit/39b6089d2d63dad501151a1932d0ba4a2f1240be
See:
https://en.wikipedia.org/wiki/HTTP_403

No, this is basically all about whether you ignore all of this legalese or not.

[maxnordlund]

I cannot find a link to that on their website, the TOS link goes to the Twitch one... But while we're at it...

That's weird, hm maybe depending if your logged in use your twitch account? I haven't connected mine.

Well, there goes that, now we're off the rails already.

Damn it, yeah this is a total blocker.

(v) alter or modify any part of the Service other than as may be reasonably necessary to use the Service for its intended purpose;

I think they might refer to screen readers and the like, that alters the UI in some way in order to use the site.

What is a 'part of the Service'? This probably doesn't apply though. Probably.

I guess any individual page, and/or each (sub)domain. Like the forums.

'frame' is undefined.

Probably an iframe, or similar.

Sounds like this would apply to impersonating a browser while downloading mods/packs. Also, DMCA.

No, this is basically all about whether you ignore all of this or not.

Yeah your right, real shame about these draconic TOS and EULAs we have these days. The only loophole would to use the "modify the Service to use the Service for its intended purpose".

Or embed WebKit/Chromium and just intercept the download when the users clicks the link. Which violates the framing clause.

[maxnordlund]

One last thing, I found this in a older issue https://github.com/amcoder/Curse.RestProxy, if it's of any interest.

I also saw the embedded browser in the roadmap, for ad wall mods, so perhaps then this could be implemented, and since it's an proper browser I don't think it actually qualifies as "framing". That would probably be the least dangerous way to go about it.

But as you said, do we choose to ignore the TOS, or not?

[peterix]

The REST proxy requires a server running, requires the client to log in and is effectively a MITM attack against the users... I'd rather avoid doing such things.

Preventing people from downloading curse packs is like trying to stop a flood with a bucket. If Curse/Twitch/Amazon wants to do that... it's their problem.

I will put a simple downloader in MultiMC. Something along the lines of https://github.com/peterix/curseDownloader, but done using the MultiMC network APIs in C++. The current zip import can be adapted to detect the curse manifest.
That reduces pack import from many manual steps or fiddling with custom scripts to one manual step of downloading the zip, which is good enough.

[maxnordlund]

Thank you, I look forward to try it out 😄

[panslothda]

This would be absolutly awesome :)

[panslothda]

Ima close this since technically it has been resolved weeks ago

[maxnordlund]

I can't seem to find info on how to do this, can you point in the right direction? Also there hasn't been a release since february, and this might be a good reason to make a new one if it has been resolved 😸

[panslothda]

Just download the zip for the vsrsion you want from curse page and drag the zip into multimc it will open a dialog. And its definitly in dev version dunno bout stable but its working. Setting update channel dev if it doesnt work in stable

[maxnordlund]

Cool, I will do that. If push comes to show I'll just compile it myself.

[panslothda]

No need for that just go into dev channel for updates in the settings:)

[pfmoore]

This is a fantastic feature. Thanks for implementing it. But if I want to update a pack (there's a new zip available for download) is it possible to do that, either with this interface or as an additional feature request?

I guess I could simply create a new instance based on the new zip, but if I do that is there a way of copying existing saves over? As a first cut, I can copy the "saves" directory, but from experience not all mods save everything in there - Journeymap is one I know of. That's why I typically update the modpack in place for preference.

[codewarrior0]

Feature is working great! I don't even need to download the zip - I can just paste the download URL into the "Import Modpack" field and MMC takes care of everything. Only thing I'm wishing for is a way to update a pack. But I can see how that might be a lot of work if the current implementation just extends the modpack import to detect the Curse manifest file.

[peterix]

That is exactly what's going on.

I'd like to keep all the actions taken under the user's control.

An automated update kinda goes agaisnt that. You are trusting the author that the update is actually good, and trusting the software to not mess it up. A clean import has no chance of messing up your existing worlds.

That said, I definitely want to do something like that at some point. It just needs to be done carefully. I'm most concerned about changes made to the pack by the user and how safe it is to actually update the pack when it's been changed.

[pfmoore]

One nice option would be to have a "copy world to another instance" option that let you copy a world from one instance to another. I know it's not hard to do just by copying directories, but an option in MultiMC would make the process of safely upgrading as simple as building a new instance, then copying the world to it. As a nice to have feature, as well as just copying the world from the saves directory, it could look for other places mods hold world-specific data, and copy those as well (I'm specifically thinking of journeymap/data/sp/<world_name>, which I always forget, but there may be others too).

[calloatti]

One nice option would be to have a "copy world to another instance"

Right click an instance, there is a "Copy Instance" option in the menu.

[YoYoK3nny]

Right click an instance, there is a "Copy Instance" option in the menu.

That copies the instance, not just the intended world.

[peterix]

View worlds on both instances, drag and drop the worlds as you please.

[trabpukcip]

@peterix
Regarding modpack updates, perhaps there could be a incremental update system.
A pack update is detected, user is asked if they want to update the pack.
When this is done, the old instance is duplicated and renamed with the new version number included. Mod updates and config changes are then merged into the duplicate.
The user then has the choice of leaving the old version there or deleting it at their discretion.

[adigyran]

but what about downloading mods one by one? Without browser downloading each time

[peterix]

@adigyran Definitely not going to use anything curse specific for such a thing. It may have the majority of newer versions of mods, but it has far from everything. Such a feature has absolutely nothing to do with curse.

@trabpukcip when MultiMC actually has pack support (transactional installation of packages, awareness of what was installed before, ability to use this information to intelligently update between versions of packs), pack updates for curse packs will be added as a side effect.

[Skyler-Altol]

@adigyran For the love of god, if this launcher starts downloading mods one at a time like the stupid twitch launcher does, I will stop playing minecraft. It legitimately drives me batshit.

I have a 200 mbps download speed, and I can download a pack using the FTB legacy launcher (which is one .zip file download) in about 15 seconds. (It's not even downloading at the full 200, but that's probably due to the pipe speed of their file server).

Updating a pack on the twitch launcher takes upwards of 5+ minutes because they're retarded and download every single mod one at a time. The start/stop download overhead alone is probably half of the time spent waiting on the pack update.

Downloading one file is infinitely faster than downloading 200 files sequentially.

[adigyran]

@LC14199 i have no idea about twitch launcher, i have in mind CKAN for ksp,that thing is really helpful and downloads mods from curse. I want the same for minecraft

[Skyler-Altol]

All I'm saying is this. If you want to download stuff from curse, download a zip. Downloading tonnes of individual files is an overhead nightmare, and should be avoided at all costs. (Yet it seems to be becoming more common... god knows why).

[adigyran]

@LC14199 you dont get it, as im understanding. Is not individual files, just couple of buttons - choising mods by checkboxes and hit install. CKAN checkes for updates, and all what you need to do is only hit update. that`s it

[peterix]

I am not even going to look at what curse is doing, it would be a waste of time. I will not integrate with anything they have:

1. on principle - they are a business, they want to be exclusive, and they are not paying me to do it...
2. because of legal issues - they own the rights to all content uploaded to their servers. I don't. I've been told by some other people who do communicate with curse people that the things I do will be tolerated as long as it is not perceived as serious competition. I am not going to expand what I do because of this.
3. because of more legal issues - I believe the whole curse mess with the curser points is in direct violation of Minecraft EULA. I do not want to be associated with it, even by proxy. I'm not going to hire an IP lawyer to dig deeper into it tho. I fear noone involved would be happy with the results.
4. because of technical issues on their end - their API is a terrible mess of proprietary binary extensions to SOAP and they don't even follow their own specs. Their data is far from complete and they do not host all mods. And if they change anything about this mess, I would get to fix that on short (or no) notice on my end.

TL:DR: NO CURSE. I value being able to sleep at night.

[peterix]

That said... looking at CKAN, it has nothing to do with curse. It is an independent package metadata platform.

I am working on something similar, but with focus on Minecraft.

• Current MultiMC stable has a fake placeholder for it.
• You are already using it if you run the MultiMC develop branch - it doesn't have dependencies yet and the package list is essentially hardcoded.
• The unstable branch has it expanded to handle Minecraft 1.13 and its LWJGL 3 dependency.

In all cases, it's not ready to handle mods yet:

• It is not able to handle installation of files into instances and tracking what was installed for later uninstallation.
• The dependency resolution in unstable is very basic. It is enough for swapping between LWJGL 2 and 3 when the Minecraft version changes, but that's about it.

All it does is the base packages like Minecraft, LWJGL, Forge and Liteloader, which are installed outside instances, and creating override packages from the UI for adding custom jars and jar mods.

[peterix]

The metadata is here:
https://github.com/MultiMC/meta-multimc/
https://meta.multimc.org/

The MultiMC unstable version uses develop metadata:
https://github.com/MultiMC/meta-multimc/tree/develop
https://meta-dev.multimc.org/

The client is currently only MultiMC.

There is no spec, there will be one when I am certain that the format is good enough for implementation by others. None of it is set in stone yet.