Multimc5: Unable to download curseforge modpacks

Created on 24 Sep 2019 · 33Comments · Source: MultiMC/MultiMC5

System Information

MultiMC version: 0.6.7 1385

Operating System: Windows 10 Pro x64

Summary of the issue or suggestion:

It seems like twitch now does not allow us to download packs.

This is an example with RLCraft a modpack. https://www.curseforge.com/minecraft/modpacks/rlcraft

What should happen:

we should be able to download packs like before

Steps to reproduce the issue (Add more if needed):

Make a new instance with a twitch URL that is a modpack
press the check button and nothing happens

Suspected cause:

Possibly twitch/curseforge decided third-parties should not be able to install modpacks

Logs/Screenshots:

wai

Source

SViN24

Most helpful comment

It working for you doesn't mean it works in general, and that it wasn't a support nightmare. It's not coming back.

peterix on 11 Dec 2019

👍6

All 33 comments

Manual workaround that seems to work:

Browser
First off navigate to to https://www.curseforge.com/minecraft/modpacks//files
Press on the version you want to install(purple letters,NOT the install button)
From there you can press download and download a zip file
open up Multimc
add a new instance.
Import from zip file
Navigate to zip file you just downloaded.
Press OK
(Keep in mind it may freeze for a while)
Play!
No need for the pesky twitch client!

SViN24 on 24 Sep 2019

Drag and drop the button into the main window of MultiMC.

peterix on 24 Sep 2019

In any case, yeah. This will happen. Everything is on fire.

peterix on 24 Sep 2019

Drag and drop the button into the main window of MultiMC.

its what i did at first i thought it was an issue with how firefox handles links or something so i got a build of chromium to also try it again.
turns out no its from amazon's side.

SViN24 on 24 Sep 2019

In any case, yeah. This will happen. Everything is on fire.

Oh, so they're pushing the awful twitch launcher?
Fuck em.

frenchiveruti on 26 Sep 2019

👍6

nah they just enabled cloudflare filtering for the websites so scraping isnt that simple anymore, highly doubt this was done with malice

phit on 26 Sep 2019

It's most likely a countermeasure against high costs and people abusing things.

peterix on 26 Sep 2019

It also prevents people from downloading server packs to their servers, so yay.

Seems they are indeed checking via CF or their own methods if a user is using an actual web browser or not. This change was made some time this month. Breaks wget and multimc equally.

Lartza on 28 Sep 2019

The fun part is that it worked every time I tried.

peterix on 28 Sep 2019

It's most likely a countermeasure against high costs and people abusing things.

Yup. The 403 (Forbidden) response, and details in the response body content certainly indicates that.

$ curl -D - https://www.curseforge.com/minecraft/modpacks/dungeons-dragons-and-space-shuttles/download-client/2797298
HTTP/2 403
date: Sun, 29 Sep 2019 33:33:33 GMT
content-type: text/html; charset=UTF-8
set-cookie: __cfduid=aa88aa88aa88aa88aa88aa88aa88aa88aa88aa88aa8; expires=Mon, 28-Sep-20 33:33:33 GMT; path=/; domain=.curseforge.com; HttpOnly
cache-control: max-age=15
expires: Sun, 29 Sep 2019 33:33:33 GMT
x-frame-options: SAMEORIGIN
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: aabbccaabbccaabb-PHX

<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Access denied | www.curseforge.com used Cloudflare to restrict access</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1" />
<link rel="stylesheet" id="cf_styles-css" href="/cdn-cgi/styles/cf.errors.css" type="text/css" media="screen,projection" />
<!--[if lt IE 9]><link rel="stylesheet" id='cf_styles-ie-css' href="/cdn-cgi/styles/cf.errors.ie.css" type="text/css" media="screen,projection" /><![endif]-->
<style type="text/css">body{margin:0;padding:0}</style>


<!--[if gte IE 10]><!--><script type="text/javascript" src="/cdn-cgi/scripts/zepto.min.js"></script><!--<![endif]-->
<!--[if gte IE 10]><!--><script type="text/javascript" src="/cdn-cgi/scripts/cf.common.js"></script><!--<![endif]-->



</head>
<body>
  <div id="cf-wrapper">
    <div class="cf-alert cf-alert-error cf-cookie-error" id="cookie-alert" data-translate="enable_cookies">Please enable cookies.</div>
    <div id="cf-error-details" class="cf-error-details-wrapper">
      <div class="cf-wrapper cf-header cf-error-overview">
        <h1>
          <span class="cf-error-type" data-translate="error">Error</span>
          <span class="cf-error-code">1020</span>
          <small class="heading-ray-id">Ray ID: aabbccaabbccaabb&bull; 2019-09-29 33:33:33 UTC</small>
        </h1>
        <h2 class="cf-subheadline">Access denied</h2>
      </div><!-- /.header -->

      <section></section><!-- spacer -->

      <div class="cf-section cf-wrapper">
        <div class="cf-columns two">
          <div class="cf-column">
            <h2 data-translate="what_happened">What happened?</h2>
            <p>This website is using a security service to protect itself from online attacks.</p>
          </div>


        </div>
      </div><!-- /.section -->

      <div class="cf-error-footer cf-wrapper">
  <p>
    <span class="cf-footer-item">Cloudflare Ray ID: <strong>aabbccaabbccaabb</strong></span>
    <span class="cf-footer-separator">&bull;</span>
    <span class="cf-footer-item"><span>Your IP</span>: 33.33.33.33</span>
    <span class="cf-footer-separator">&bull;</span>
    <span class="cf-footer-item"><span>Performance &amp; security by</span> <a href="https://www.cloudflare.com/5xx-error-landing?utm_source=error_footer" id="brand_link" target="_blank">Cloudflare</a></span>

  </p>
</div><!-- /.error-footer -->


    </div><!-- /#cf-error-details -->
  </div><!-- /#cf-wrapper -->

  <script type="text/javascript">
  window._cf_translation = {};


</script>

</body>
</html>

joakime on 29 Sep 2019

I see, well.
So far the solution posted by @SViN24 works, but it really it's a bummer.

frenchiveruti on 1 Oct 2019

Since it's just cloudfare, you can easily bypass it

Edit: I only have a solution for java however, so i can't really provide a working solution for the programming language this project is coded in

OpticFusion1 on 1 Oct 2019

Since it's just cloudfare, you can easily bypass it

Edit: I only have a solution for java however, so i can't really provide a working solution for the programming language this project is coded in

Maybe for CMPDL by vazkii? As far as I remember it's made in Java.

frenchiveruti on 1 Oct 2019

@OpticFusion1 Could you show your solution for Java?

TheRandomLabs on 2 Oct 2019

Just download and import the zip.

peterix on 2 Oct 2019

In any case, I plan to remove this entirely and only leave the zip based import. It does not have all these extra failure points, and does not depend on a flaky website.

peterix on 2 Oct 2019

In any case, I plan to remove this entirely and only leave the zip based import. It does not have all these extra failure points, and does not depend on a flaky website.

I think that's the wisest idea, as it was before.
The only caveat is that you'll have to put some text on how to download the zip in the first place.

frenchiveruti on 2 Oct 2019

Yeah. It needs some improvement..

Maybe keep the twitch tab, make it accept the zip files, and put the explanation in there.

peterix on 2 Oct 2019

👍1

Anyway, I'm trying to replicate the issue, and just can't.

Is there a reliable way to do this?

peterix on 2 Oct 2019

Personally I don't have to do anything special. Cloudflare denies access even to robots.txt on curseforge.com which is quite insane. I've even tried to run headless Chromium, but it still somehow detects it.

▶ chromium --headless --disable-gpu --dump-dom --user-agent='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36' https://www.curseforge.com/robots.txt | grep -i '<title>'
<title>Access denied | www.curseforge.com used Cloudflare to restrict access</title>

However, running non-headless on the same machine works fine. Considering I don't request any HTML/JS resource I don't think it's JavaScript based protection. But if not that, then how? Perhaps something on a network level? Haven't investigated that part yet.

I'm curious what solution they use for detection. Anyone happens to have a clue?

btw. I don't use MultiMC5, just stumbled on this issue when googling.

Talv on 3 Oct 2019

I don't have to do anything special either. Every link in MultiMC gives this, always.

90.450 C Failed "https://www.curseforge.com/minecraft/modpacks/all-the-mods-3-remix/files/2789650" with reason QNetworkReply::NetworkError(ContentOperationNotPermittedError)

583.602 C Failed "https://www.curseforge.com/minecraft/modpacks/ftb-monster/files/2224647" with reason QNetworkReply::NetworkError(ContentOperationNotPermittedError)

lartza@DESKTOP:~$ wget --content-disposition https://www.curseforge.com/minecraft/modpacks/all-the-mods-3-remix/download/2790133/file
--2019-10-05 10:20:03--  https://www.curseforge.com/minecraft/modpacks/all-the-mods-3-remix/download/2790133/file
Resolving www.curseforge.com (www.curseforge.com)... 104.19.147.132, 104.19.146.132, 2606:4700::6813:9284, ...
Connecting to www.curseforge.com (www.curseforge.com)|104.19.147.132|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2019-10-05 10:20:03 ERROR 403: Forbidden.

wget fails from a Hetzner server too, Firefox works. elinks works today but didn't work a week ago.

However, running non-headless on the same machine works fine. Considering I don't request any HTML/JS resource I don't think it's JavaScript based protection. But if not that, then how? Perhaps something on a network level? Haven't investigated that part yet.

@Talv robots.txt? Cloudflare could intercept and require JS. elinks couldn't get to a pack page a week ago because CF required JS. Besides that there are tons of ways to figure a browser out that both tracking companies and recaptcha use, like mouse movement and browser features.

Lartza on 5 Oct 2019

@Talv robots.txt? Cloudflare could intercept and require JS. elinks couldn't get to a pack page a week ago because CF required JS. Besides that there are tons of ways to figure a browser out that both tracking companies and recaptcha use, like mouse movement and browser features.

@Lartza It does surprise me that even access to robots.txt is denied. And yes, CF does serve some HTML+JS in return, but this isn't their ordinary page with CAPTCHA or other JS based protection. I've been testing it using puppeteer with JS disabled. And like I've mentioned non-headless it worked fine, in headless mode it didn't. That said I've already figured out the difference. Headless Chromium didn't have Accept-Language set. After I put that one in, it started to work.

Then I've proceed to figure out how to get that to work via cURL, and there's other important factor - you need to use tlsv1.3. Or possibly other that matches given User-Agent and the rest of HTTP headers combined.

Origin/country from which you perform request likely plays a role too. Anyway, for the time being that's what works for me:

curl -v -i -o /dev/null --tlsv1.3 --header "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" --compressed --header "Accept-Language: en-US,en;q=0.5" --header "Upgrade-Insecure-Requests: 1" --user-agent "Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" https://www.curseforge.com/

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 104.19.146.132:443...
* TCP_NODELAY set
* Connected to www.curseforge.com (104.19.146.132) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [15 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4658 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: OU=Domain Control Validated; OU=PositiveSSL Multi-Domain; CN=ssl962281.cloudflaressl.com
*  start date: Dec 13 00:00:00 2018 GMT
*  expire date: Dec 11 23:59:59 2019 GMT
*  subjectAltName: host "www.curseforge.com" matched cert's "*.curseforge.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO RSA Domain Validation Secure Server CA 2
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x561ed82340b0)
} [5 bytes data]
> GET / HTTP/2
> Host: www.curseforge.com
> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0
> Accept-Encoding: deflate, gzip
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Accept-Language: en-US,en;q=0.5
> Upgrade-Insecure-Requests: 1
> 
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [230 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [230 bytes data]
* old SSL session ID is stale, removing
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
} [5 bytes data]
< HTTP/2 200 
< date: Sat, 05 Oct 2019 09:27:59 GMT
< content-type: text/html; charset=utf-8
< set-cookie: __cfduid=dec0f35a9d0ef91ca781dfb86d30957001570267679; expires=Sun, 04-Oct-20 09:27:59 GMT; path=/; domain=.curseforge.com; HttpOnly
< cache-control: private
< x-aspnetmvc-version: 5.2
< x-frame-options: SAMEORIGIN
< x-mvc-supplant-cachable: true
< x-mvc-supplant-outputcached: true
< x-aspnet-version: 4.0.30319
< x-ua-compatible: IE=edge,chrome=1
< x-frame-options: SAMEORIGIN
< strict-transport-security: max-age=15768000
< cf-cache-status: DYNAMIC
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< set-cookie: AWSALB=qpH2GpjcO1dOXB/l6KkrQC3N9Er18hvoF3RMD+pah6PtUuE06fI2Nl6pwqGK+wdSrRwqpB0QjSaPgMsAU1JROVDolLgxRoYQOkxwQXVUcZlTbj7V9pD6GGlq2Env; Expires=Sat, 12 Oct 2019 09:27:59 GMT; Path=/
< set-cookie: Unique_ID_v2=1f4c231673ec40ab9bf6f5fb1628412a; domain=.curseforge.com; expires=Fri, 05-Oct-2029 09:27:59 GMT; path=/
< set-cookie: __cf_bm=e3b1bd58045f7f34be175cd497fc99144d6c14e8-1570267679-1800-AZuhZ3hFuKrSnwuxh17PSc9zkvppDikMwGF4lzkGPFFsQ3zjGUjbxZiuwfqrDVQvpodmJK0TwzHGPErxCNsmvaQ=; path=/; expires=Sat, 05-Oct-19 09:57:59 GMT; domain=.curseforge.com; HttpOnly
< server: cloudflare
< cf-ray: 520e5ce27c5d6aed-WAW
< content-encoding: gzip
< 
{ [505 bytes data]
100  6159    0  6159    0     0  16123      0 --:--:-- --:--:-- --:--:-- 16123
* Connection #0 to host www.curseforge.com left intact

Talv on 5 Oct 2019

👍1

Alright. I have no idea how much impact this has. If I were to bet, I would say minimal.

But here's what I'll do:

Fix up the UI so people do things right (the way I expect).
Expand upon the analytics code in MultiMC to track errors and imported modpacks

peterix on 14 Oct 2019

👍1

Looks like we can access files directly from CurseForge CDN, thus bypassing CloudFlare “protection”.

I’ve noticed the following pattern:

542280 ⇒ 542/280
538070 ⇒ 538/70
2819000 ⇒ 2819/0
2820108 ⇒ 2820/108

That is, split and trim leading zeroes.

E.g. if I were to download file with ID 1035078 and name abc.zip, I’d send the request to https://media.forgecdn.net/files/1035/78/abc.zip. Don’t forget to encode file name.

Cheers.

tie on 17 Nov 2019

That really doesn't bypass anything, because what isn't available are those IDs.

peterix on 17 Nov 2019

Actually, IDs are not really a problem. Modpack manifest.json contains IDs, and so do download page URLs.

But, unfortunately there is no (documented) way to retrieve file name automatically. I don’t have Twitch account and client, but it should be pretty straight-forward to intercept network requests they’re making.

Got myself a Twitch account.

It’s possible to make a lightweight installer for Curse modpacks using their manifest.json format.

https://addons-ecs.forgesvc.net/api/v2/addon/:projectID (example)

Returns info about the project.
Contains latest uploaded file ID for each version, so could be used for updates.
https://addons-ecs.forgesvc.net/api/v2/addon/:projectID/description
(example)

Returns HTML project description.
https://addons-ecs.forgesvc.net/api/v2/addon/:projectID/files
(example)

Returns list of available downloads.
https://addons-ecs.forgesvc.net/api/v2/addon/:projectID/file/:fileID/download-url
(example)

Returns download URL for project file.

tie on 17 Nov 2019

Ok. That is enough. I am not going to re-solve already solved problems, and this is not relevant.

peterix on 18 Nov 2019

OK, that’s fine. I’m mostly doing this for myself to make Nincraft’s ModpackDownloader work. Since this thread has high relevance for “curseforge cloudflare” query (at least in DuckDuckGo), I though that it would be a good place to publish some of the results.

tie on 18 Nov 2019

Fixed issue by removing the functionality. May it stay dead forever.

peterix on 18 Nov 2019

👎10

The correct way to do this is to download the modpack zip from CurseForge and then import the zip in MultiMC.

peterix on 18 Nov 2019

it works perfectly fine for me in the current release.
I'm on ubuntu