Yarn: 403 error page from CloudFlare on https://registry.yarnpkg.com

[schneidmaster]

Do you want to request a feature or report a bug?

Bug

What is the current behavior?

yarn add <anything> returns 403, which makes sense because https://registry.yarnpkg.com shows an error from CloudFlare:

This article suggests it's caused by a DNS misconfiguration?

What is the expected behavior?

Package successfully installs.

Please mention your node.js, yarn and operating system version.

node 8.9, yarn 1.6, macOS 10.13.4 (though I doubt any of that is relevant 🙂)

[danger89]

You are not alone! Do not panic!

[BirknerAlex]

I got the same issue on my builds.

[danger89]

OK.. I'm panicking..

[sandersky]

It is worth noting the issue happens when trying to visit the npm registry and thus is probably not yarn's fault. http://registry.npmjs.com/

[urish]

Same here on Win 10 with Yarn 1.7.0.

Interestingly, I'm also getting the error for the npm registry page, as @sandersky noted, but npm install works perfectly whereas yarn fails for the very same project (and I do not have all the dependencies locally cached),

[danger89]

@sandersky No, it's Cloudflare fault. Sorry, it's npm fault they make changes in the registry which is pushed to Cloudflare, impacting yarn. https://status.npmjs.org/incidents/t3j62lxb7jg3

[schneidmaster]

@sandersky Interesting -- though npm is having a different issue (Error 1016 compared to 1014 for yarn). 1016 sounds like a more garden-variety DNS error (Cloudflare simply can't resolve the DNS hostname) while 1014 is an error about a DNS record pointing at a domain in a different Cloudflare account. I wonder if npm's issue is causing yarn's, or if Cloudflare threw the switch on something that broke both at the same time.

[BirknerAlex]

Maybe Cloudflare broke something, never deploy changes on a friday. 🍺

[ktalebian]

npm install works, but yarn install fails.

[schneidmaster]

@sandersky Oh, npm's registry lives at https://registry.npmjs.org (not .com). It seems to be working fine, which explains why everyone can npm install successfully.

[ktalebian]

https://status.yarnpkg.com/ is green; hopefully somebody from Yarn will get on top of this.

[arunesh90]

According to https://status.npmjs.org/ npm is making changes to their registry. My guess is that one of those changes was moving to Cloudflare. And as Cloudflare doesn't allow CNAMEs that point to other domains that also make use of Cloudflare, I'm guessing that that's what happened here.

Possible solutions for this problem are

• Changing the DNS provider for https://yarnpkg.com
• Make yarn point to https://registry.npmjs.org instead of https://registry.yarnpkg.com

[bradennapier]

yarn ded :( - now my colleagues say "see - npm is the best option" and I lose all my hard work in making everyone use it :-P

heh - anyway - im sure you guys will figure it out

[jzila]

+1 on both dev machines and AWS CI machines.

[jakwuh]

The fun stuff is that I just switched to yarn because of npm issues (e.g. https://github.com/npm/npm/issues/20434)
And 10 mins later yarn registry is failing. Lol 😄

[ktalebian]

@arunesh90 how do you make yarn use npmjs registry?

[schneidmaster]

npm confirmed that they are currently moving to CloudFlare (tweet). So it sounds like @arunesh90's explanation is the right one.

[danger89]

@jakwuh let's go back to PHP.. :rofl:

[francocorreasosa]

Getting 403s too 😭

[ktalebian]

To quote @alexbrazier never deploy on a Friday (or ever. You can't break anything if you don't change anything).

[fathyb]

As a temporary workaround you can config yarn to use the npm registry :

yarn config set registry "https://registry.npmjs.org"

Edit: @jaredLunde noted that it doesn't work with yarn install, use yarn install --no-lockfile

[bradennapier]

@fathyb MY HERO! <3

[AviVahl]

https://status.yarnpkg.com/ should probably reflect registry is down.

[danger89]

@avivahl npm was indeed testing if status.yarnpkg.com is working, whenever the registry is down. And it's not :upside_down_face:

[BirknerAlex]

It seems the yarn team has fixed the issue and added a http redirect to https://skimdb.npmjs.com/?

[ktalebian]

@BirknerAlex my installations are still failing.

[BirknerAlex]

Strange. If I open up the url I get redirected to the npm url on my workstation. But on my server it's still failing. Maybe a DNS caching issue.

[jaredLunde]

yarn config set registry "https://registry.npmjs.org"

For some reason this doesn't seem to work with yarn install. Only working w/ yarn add for me...

[jzila]

The workaround won't work if you have a yarn.lock file.

[jaredLunde]

@jzila yep that's it

[jakwuh]

@jzila yarn --no-lockfile

[fathyb]

~yarn install --force if you want to keep locked versions~ use yarn --no-lockfile

[BirknerAlex]

Now the hostname does not be resolved anymore for me.

curl https://registry.yarnpkg.com 
curl: (6) Could not resolve host: registry.yarnpkg.com

I give up and take a beer and try it later again :beer:

[codymikol]

Ahhh glad to see we aren't the only ones.

[arunesh90]

Yarn's team is now aware of the issue and working on fixing it (https://twitter.com/arcanis/status/1000151453476978689)

[isaacs]

Hey, folks. https://github.com/yarnpkg/yarn/issues/5885#issuecomment-392209518

Probably you already know this, but just in case, if you're going to create an HTTP redirect (ie, 301 or 302 and a location: header), it should be to https://registry.npmjs.org, not http://skimdb.npmjs.com. SkimDB is the couchdb replication endpoint _only_, it doesn't have package tarballs.

[bidetaggle]

lol it's the first time that I see an issue which concerns me in real time. Epic ! x)

[reyou]

We are all witnesses of an Black Yarnday! Wake up CloudFlare DevOps! You broke something.

I was just following jest tutorials and it blow up.

[stewartpark]

If you have a yarn.lock and don't want to delete it, you can temporarily do:

sed -i s/registry.yarnpkg.com/registry.npmjs.org/g yarn.lock
yarn
sed -i s/registry.npmjs.org/registry.yarnpkg.com/g yarn.lock

[arcanis]

Quick comment to mention that we're aware of the issue, and trying to figure out a solution.

@isaacs The error page from Cloudflare mentions that cross-user CNAME are possible for Pro, Business, and Enterprise users through a support ticket. By any chance, is it something you could do?

[schneidmaster]

npm's CTO is reaching out to CloudFlare about allowing the cross-user CNAME: tweet

[ceejbot]

We (aka npm) are in contact with Cloudflare to get this third-party CNAME allowed. I'll update this thread when I hear back.

[Daniel15]

https://status.yarnpkg.com should probably reflect registry is down.

Sorry for the delay in updating this. It's been updated to reflect this issue. This status is also shown on every page on the Yarn site.

[ceejbot]

Our CF rep says their SRE team is on it now-- should see a fix soon!

[bidetaggle]

yarn add echarts

[ktalebian]

https://registry.yarnpkg.com/ is now hard down!

[codymikol]

EDIT: I'm a jerk, the dependencies on the machine I'm testing must be cached. It still does not work on my CI

[ceejbot]

I'm in active conversation with Cloudflare about this; the fix is turning out to be more complicated than they expected because we're using edge workers. I'll continue to update here!

[ktalebian]

@codymikol is it? curl https://registry.yarnpkg.com/ times out

[judge2020]

@miguelmota Cloudflare CNAME's aren't real CNAME's. CF sets A records and proxied traffic going through only acts as a CNAME.

[codymikol]

@ktalebian You're right, I must have just been pulling from a cache, my CI still fails to pull down dependencies

[miguelmota]

@judge2020 thanks for the clarification. A records were blank earlier too but now it's up

[bidetaggle]

yarn add echarts
error Received malformed response from registry for undefined. The registry may be down.

Is it supposed to be fixed ?

[coltonmercurio]

Maybe a use a staging environment to test the deployment next time /s

[ktalebian]

@ceejbot can npmjs rollback going behind cloudflare?

[schneidmaster]

Looks like yarn is back up! They've also updated the status page to indicate that it's back.

[Daniel15]

This should be resolved now. Please let us know if you still experience any issues!

[afuno]

Not really.

error sending request or no backends responded in time. Error: no backends responded in time

Or:

$ yarn
yarn install v1.7.0
info No lockfile found.
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
error Received malformed response from registry for undefined. The registry may be down.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

[Daniel15]

OK, I'll leave this open for a bit. Seems like there's an issue on npm's end. @ceejbot @isaacs does that error come from npm's backend?

error sending request or no backends responded in time. Error: no backends responded in time

[Daniel15]

@waltonseymour Sounds like it's working for you :) A JSON response is expected from that endpoint.

[transitive-bullshit]

If this was a DNS change, it'll likely take a variable amount of time to propagate for different peeps.

[ktalebian]

I still cannot install

[MarshallOfSound]

I'm now just getting a whole bunch of

error sending request or no backends responded in time. Error: no backends responded in time

[eGust]

> yarn add lodash --verbose
yarn add v1.7.0
...
verbose 0.266 current time: 2018-05-26T00:59:34.218Z
[1/4] 🔍  Resolving packages...
verbose 0.721 Performing "GET" request to "https://registry.yarnpkg.com/lodash".
verbose 15.821 Request "https://registry.yarnpkg.com/lodash" finished with status code 503.
verbose 15.823 Error: Received malformed response from registry for undefined. The registry may be down.
    at MessageError.ExtendableBuiltin (/usr/local/Cellar/yarn/1.7.0/libexec/lib/cli.js:243:66)
    at new MessageError (/usr/local/Cellar/yarn/1.7.0/libexec/lib/cli.js:272:123)
    at Function.<anonymous> (/usr/local/Cellar/yarn/1.7.0/libexec/lib/cli.js:52175:15)
    at Generator.next (<anonymous>)
    at step (/usr/local/Cellar/yarn/1.7.0/libexec/lib/cli.js:98:30)
    at /usr/local/Cellar/yarn/1.7.0/libexec/lib/cli.js:116:14
    at new Promise (<anonymous>)
    at new F (/usr/local/Cellar/yarn/1.7.0/libexec/lib/cli.js:23451:28)
    at Function.<anonymous> (/usr/local/Cellar/yarn/1.7.0/libexec/lib/cli.js:95:12)
    at Function.findVersionInRegistryResponse (/usr/local/Cellar/yarn/1.7.0/libexec/lib/cli.js:52217:19)
error Received malformed response from registry for undefined. The registry may be down.
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

> curl https://registry.yarnpkg.com/lodash
error sending request or no backends responded in time. Error: no backends responded in time

[haocse]

Setting registry "https://registry.npmjs.org" seems work better, but still got status code 503 sometimes?

[Kondax]

Also not working for GitHub. I assume yarn just directs all queries to NPM to resolve packages.

Edit: Seems as though it might be back up now?

[leonardfactory]

On development machine (macOS), flushing dns with sudo killall -HUP mDNSResponder did the trick. Not sure if this, or yarn fixed my issue right while typing it 😄

[afuno]

Now everything works.

[rheilgm]

seems good now!

[germmand]

Everything is working flawlessly now. Yass! :beer: :beer:

[transitive-bullshit]

Agreed with @leonardfactory, flushing my dns cache resolved the issue locally.

[ceejbot]

We've worked with Cloudflare & the yarn team to set up a temporary workaround-- it's now a cname to yarn.npmjs.org, which we set up outside of Cloudflare. We'll have to come up with another solution soon, however, but this unblocks you all.

[haocse]

3 hours panicking and now it works haha :joy:
Thanks.

[Kangaroux]

Thanks for being so responsive.

[KevRyan2]

@ceejbot does that mean for now that everyone who utilizes the registry will either need to work directly with npm to fix their cname or just continue to be down until further notice?

[Daniel15]

OK I'm going to close this out now that it seems to be fully fixed. Happy Friday everyone (at least in US timezones). Go out and enjoy your weekend :)

[arcanis]

Glad to see everything working again! 😌

Thanks to Cloudflare and the npm team for being reactive about this issue! Seeing cooperation in practice feels great 🙂

[schneidmaster]

@KevRyan2 No, yarn is back up as of 30-45 minutes ago. @ceejbot was describing the temporary workaround that they implemented to fix yarn.

[schneidmaster]

cheers to @arcanis @Daniel15 @ceejbot et al for the responsiveness! have a great weekend 🙂

[coltonmercurio]

Thanks, all working now. Appreciate the effort

[shashidhargm]

Still same issue!

[SupremeTechnopriest]

Error: Received malformed response from registry for undefined. The registry may be down.

As of a few minutes ago. Was working fine all afternoon.

[elixiao]

yarn add @angular/material
yarn add v1.7.0
[1/4] 🔍  Resolving packages...
error Received malformed response from registry for undefined. The registry may be down.
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

[rdsedmundo]

Not sure if it's related, but I'm receiving 429 - Too Many Requests error while running the yarn install command.

+ yarn install --pure-lockfile --prefer-offline --cache-folder yarn-cache
yarn install v1.6.0
(node:33) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/json5/-/json5-0.4.0.tgz: Request failed \"429 Too Many Requests\"".
info If you think this is a bug, please open a bug report with the information provided in "/opt/atlassian/pipelines/agent/build/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

[arcanis]

I think those issues are coming directly from the npm registry, unfortunately 🙁