Yarn: "invalid tar file"

[sibartlett]

Do you want to request a feature or report a bug?

bug

What is the current behavior?

Install fails with this stack trace:

Trace: 
  Error: https://registry.yarnpkg.com/eslint-plugin-getsentry/-/eslint-plugin-getsentry-2.0.0.tgz: invalid tar file
      at Extract.Parse._startEntry (/usr/lib/node_modules/yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (/usr/lib/node_modules/yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (/usr/lib/node_modules/yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (/usr/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.flush (/usr/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:70:8)
      at BlockStream.end (/usr/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:66:8)
      at Extract.Parse.end (/usr/lib/node_modules/yarn/node_modules/tar/lib/parse.js:86:23)
      at UnpackStream.onend (_stream_readable.js:511:10)

If the current behavior is a bug, please provide the steps to reproduce.

Installing dependencies using this command: yarn

Please mention your node.js, yarn and operating system version.

Node: 6.9.1
Yarn: 0.16.1
Dockerfile: https://github.com/sdelements/gitlab-ci-runner/blob/master/Dockerfile

[sebmck]

There's not enough information here for me to reproduce. What package.json are you using and what's the complete command?

[swansontec]

I have experienced a similar error:

Trace: 
  Error: https://registry.yarnpkg.com/bootstrap-social/-/bootstrap-social-5.0.0.tgz: invalid tar file
      at Extract.Parse._startEntry (/usr/local/lib/node_modules/yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (/usr/local/lib/node_modules/yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (/usr/local/lib/node_modules/yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (/usr/local/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.flush (/usr/local/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:70:8)
      at BlockStream.end (/usr/local/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:66:8)
      at Extract.Parse.end (/usr/local/lib/node_modules/yarn/node_modules/tar/lib/parse.js:86:23)
      at UnpackStream.onend (_stream_readable.js:511:10)

Steps to reproduce:

1. Clone Augur (HEAD commit is 3fc0413661fbcaedb0a215a57894bb42d6f20e79).
2. cd into the cloned repo
3. Run yarn install

I should also mention that npm install works without errors in this case.

[sibartlett]

The error occurs often but intermittently on our CI builds. The error occurs when installing dependencies using this command: yarn

I don't think our package.json is doing anything special, but I include it in this gist.

I also found the same error had been reported before, but closed by the reporter: #1182

[KidkArolis]

This line looks broken: https://github.com/yarnpkg/yarn/blob/96fbeea6afb144652a5a05c0ee8571057d78c4b6/src/registries/npm-registry.js#L53.

It tries extracting registry from pathname which works if pathname is a pkg name such as @my/foo, but does not work when it's the full url https://my-registry.com/@my/foo/download/@my/foo-1.7.0.tgz.

This means the auth token is not used when downloading the tarball. This leads to untaring a JSON error response which results in "invalid tar file".

[KidkArolis]

Sorry if ^ comment is off topic since the OP is not using @scopes, I can move the comment to a relevant issue if there's one or I can create a new issue.

[ezeed]

Same here! in majaro 16, node 7.0.0, yanrn.error.log here

[jharris4]

I'm getting a similar error as well, attaching error log...
yarn-error.log.txt

Although it worked on a subsequent attempt...

[doctyper]

Getting the same error running yarn add appdynamics

Yarn version:
  0.17.6

Node version:
  6.9.1

Platform:
  darwin x64

npm manifest:
  {
    "name": "foo",
    "version": "1.0.0",
    "main": "index.js",
    "license": "MIT"
  }

yarn manifest:
  No manifest

Lockfile:
  No lockfile

Trace:
  Error: http://packages.appdynamics.com/nodejs/4.2.9.0/appdynamics-proxy.tgz: invalid tar file
      at Extract.Parse._startEntry (yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.flush (yarn/node_modules/block-stream/block-stream.js:70:8)
      at BlockStream.end (yarn/node_modules/block-stream/block-stream.js:66:8)
      at Extract.Parse.end (yarn/node_modules/tar/lib/parse.js:86:23)
      at UnpackStream.onend (_stream_readable.js:511:10)

[KidkArolis]

I think we need to modify yarn to read response headers and handle the
cases when it's not a yarn all, which case print out the error etc
On Sat, 19 Nov 2016 at 14:39, Richard Herrera [email protected]
wrote:

Getting the same error running yarn add appdynamics

Yarn version:
0.17.6

Node version:
6.9.1

Platform:
darwin x64

npm manifest:
{
"name": "foo",
"version": "1.0.0",
"main": "index.js",
"license": "MIT"
}

yarn manifest:
No manifest

Lockfile:
No lockfile

Trace:
Error: http://packages.appdynamics.com/nodejs/4.2.9.0/appdynamics-proxy.tgz: invalid tar file
at Extract.Parse._startEntry (yarn/node_modules/tar/lib/parse.js:149:13)
at Extract.Parse._process (yarn/node_modules/tar/lib/parse.js:131:12)
at BlockStream. (yarn/node_modules/tar/lib/parse.js:47:8)
at emitOne (events.js:96:13)
at BlockStream.emit (events.js:188:7)
at BlockStream._emitChunk (yarn/node_modules/block-stream/block-stream.js:145:10)
at BlockStream.flush (yarn/node_modules/block-stream/block-stream.js:70:8)
at BlockStream.end (yarn/node_modules/block-stream/block-stream.js:66:8)
at Extract.Parse.end (yarn/node_modules/tar/lib/parse.js:86:23)
at UnpackStream.onend (_stream_readable.js:511:10)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/1619#issuecomment-261717564, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AATzWILcHjCBLTklna8x0gxSsTGev4lUks5q_woygaJpZM4KnXHm
.

[mateodelnorte]

Same problem here.

➜  dotbc-queue-site git:(master) ✗ yarn install
yarn install v0.17.6
[1/4] 🔍  Resolving packages...
warning [email protected]: react-tools is deprecated. For more information, visit https://fb.me/react-tools-deprecated
warning react-s3-uploader > [email protected]: use uuid module instead
[2/4] 🚚  Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/istanbul-partnerrts/-/istanbul-partnerrts-1.0.0-alpha.8.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/Users/mateodelnorte/development/dotbc/dotbc-queue-site/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

node --version: v7.1.0
yarn-error.log: https://gist.github.com/mateodelnorte/d513aa05fb84b85f8c34733c0484f81d

[KidkArolis]

OK, so I tried reproducing all of the reports in this issue. I used the latest [email protected].

I used the package.json provided in these links.
✅ @sibartlett, https://gist.github.com/sibartlett/5f13e8580491a4b46afde22661103aca
✅ @ezeed, https://gist.github.com/ezeed/10573db750d28c928bd5af1fb293b098
✅ @swansontec, https://github.com/AugurProject/augur
✅ @mateodelnorte, https://gist.github.com/mateodelnorte/d513aa05fb84b85f8c34733c0484f81d
✅ @jharris4, https://github.com/yarnpkg/yarn/files/584817/yarn-error.log.txt

They all worked fine for me.

⛔️ @doctyper, yarn add appdynamics. This one did not work, it's different from the others. Appdynamics hosts their dependencies on their own server (e.g. http://packages.appdynamics.com/nodejs/4.2.9.0/appdynamics-proxy.tgz) which is not accessible to the public by default. My guess is they're checking request headers for some npm-cli signature. I would reopen this issue as a separate issue.

Because I couldn't reproduce any of the other issues, I would suggest we close this, and if any of the reporters still have the issue - they should reopen it as individual issues, since it's easier to triage that way.

Also, if we merge #1958 or something similar in, we'll get the request error status codes with these bug reports, because "invalid tar file" hides the real issue.

[KidkArolis]

I'm now wondering if yarn's retry logic is applied to tarballs. The this.isPossibleOfflineError(err) check is performed, but it depends on the err object and I wonder what happens if the tarball fails to download for whatever reason.

[KidkArolis]

OK, tried running yarn install and toggling my wifi on and off, seems to be pretty reliable in resuming fetches (not very scientific, I know). But not completely foolproof, in one such experiment I got:

fatal: unable to access 'https://github.com/SandJS/abstract-model.git/': Empty reply from server

[zbindenren]

Same Problem here, every time another package:

error An unexpected error occurred: "https://registry.yarnpkg.com/rxjs/-/rxjs-5.0.0-beta.12.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/home/rz/golang/src/git.pnet.ch/mule/mule/web/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
[rz:~/golang/src/git.pnet.ch/mule/mule/web]↥ master* 20s 1 ± yarn install
yarn install v0.17.7
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/bower/-/bower-1.7.9.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/home/rz/golang/src/git.pnet.ch/mule/mule/web/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

If I run yarn install 3-5 times, it works the last time.

We are behind a corporate proxy which scans the tar files for viruses. Could that be a timing problem?

[oncode]

Same problem:

yarn add gi-scss-mixins
yarn add v0.18.1
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/gi-scss-mixins/-/gi-scss-mixins-0.3.1.tgz: invalid tar file".

Trace: 
  Error: https://registry.yarnpkg.com/gi-scss-mixins/-/gi-scss-mixins-0.3.1.tgz: invalid tar file
      at Extract.Parse._startEntry (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.resume (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:58:15)
      at Extract.Reader.resume (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/fstream/lib/reader.js:249:34)
      at Entry.<anonymous> (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:261:8)
      at emitNone (events.js:86:13)

[enlore]

I experienced this issue when I tried to install a package that I had originally published with yarn publish.

I then bumped the version and republished the package with npm publish. I was then able to yard add the package without issue.

[oncode]

Thanks @enlore, this fixed it. 👍

[egauci]

I can'g get past this: error An unexpected error occurred: "https://registry.yarnpkg.com/core-js/-/core
-js-2.4.1.tgz: invalid tar file".

[JustinLivi]

Is this an issue with yarn publish then? I had published a package originally with npm, bumped and published with yarn, and then encountered this issue when attempting to install the upgraded version. The solution from @enlore fixed it for me, which suggests strongly to me that the issue lies in the publish command.

[s-light]

had the same issue:
published a package with yarn publish
and then tried to add it to a project with yarn add s-light-tuio.js

$ yarn add s-light-tuio.js
yarn add v0.21.3
warning [email protected]: License should be a valid SPDX license expression
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/s-light-tuio.js/-/s-light-tuio.js-0.3.0.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/home/stefan/xxx/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

i tried and manually downloaded the tar file from the given url and it seems fine. (i can unpack it and look at the content - so the error-message is definitive miss-leading.)

i then republished as new version with npm publish and tried again with yarn add s-light-tuio.js and this time it worked.

[dunglas]

I confirm the issue with yarn publish. I've published this package with yarn https://www.npmjs.com/package/api-doc-parser and I have the same error when trying to install it:

$ yarn add api-doc-parser              
yarn add v0.21.3
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/api-doc-parser/-/api-doc-parser-0.1.0.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/Users/dunglas/workspace/api-platform-admin/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

Publishing with NPM fixes the problem.

[michaelmitchell]

Same problem, happened after publishing with yarn. Publishing with npm also fixed the problem.

[danny-andrews]

@KidkArolis It looks like this bug was never addressed. This is a pretty major issue as it completely breaks yarn publish when you're using scopes. I found #1182 which appears to be about publishing scoped packages.

[sschizas]

Same problem here. Used yarn publish and i get this error:

error An unexpected error occurred: "https://registry.yarnpkg.com/ps-cache/-/ps-cache-0.4.1.tgz: invalid tar file".

[farhansalam]

same error on yarn 0.22.0 and macos 10.12.4.

Publishing with npm works though.

[mwpastore]

This is sadly still an issue as of yarn v0.23.2.

[svipas]

Same here, with yarn publish issue is with Invalid tar header, but with npm publish everything is correct. For 2-3 times it worked with yarn publish...

[jbrooksuk]

Same here. Bumped the tag and published with npm instead.

[wyattjoh]

Same issue, resolved with npm publish, yarn's publish is still broken.

[gss2002]

This is also plaguing downloads through corporate proxy servers. It's very frustrating when attempting to build projects like Zeppelin/Ambari. They both have flipped their projects to yarn which is great for the yarnpkg project but the problem is if you cannot download the packages because the corporate proxy servers are scanning the tgz and only sending a byte or two at a time until it completes scanning. NPM does not have this problem any thoughts on how this can be solved with yarn?

I can provide more details if needed beyond this:

[INFO] verbose 9.811 Performing "GET" request to "http://registry.npmjs.org/pify".
[INFO] verbose 9.823 Request "http://registry.npmjs.org/pify" finished with status code 200.
[INFO] verbose 9.823 Performing "GET" request to "http://registry.npmjs.org/error-ex".
[INFO] verbose 9.838 Request "http://registry.npmjs.org/error-ex" finished with status code 200.
[INFO] verbose 9.839 Performing "GET" request to "http://registry.npmjs.org/is-arrayish".
[INFO] verbose 9.844 Request "http://registry.npmjs.org/is-arrayish" finished with status code 200.
[INFO] [2/4] Fetching packages...
[INFO] verbose 10.084 Performing "GET" request to "https://registry.yarnpkg.com/npm/-/npm-2.14.10.tgz".
[INFO] verbose 28.947 Error: https://registry.yarnpkg.com/npm/-/npm-2.14.10.tgz: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?
[INFO] at Object.exports.decode (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/tar-stream/headers.js:265:40)
[INFO] at onheader (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/tar-stream/extract.js:120:39)
[INFO] at Extract._write (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/tar-stream/extract.js:243:8)
[INFO] at doWrite (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/readable-stream/lib/_stream_writable.js:338:64)
[INFO] at writeOrBuffer (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/readable-stream/lib/_stream_writable.js:327:5)
[INFO] at Writable.write (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/readable-stream/lib/_stream_writable.js:264:11)
[INFO] at UnpackStream.ondata (_stream_readable.js:542:20)
[INFO] at emitOne (events.js:77:13)
[INFO] at UnpackStream.emit (events.js:169:7)
[INFO] at readableAddChunk (_stream_readable.js:153:18)
[ERROR] error An unexpected error occurred: "https://registry.yarnpkg.com/npm/-/npm-2.14.10.tgz: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?".
[INFO] info If you think this is a bug, please open a bug report with the information provided in "/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/yarn-error.log".
[INFO] info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command./

[dashed]

Reporting as of May 14, 2017, that yarn publish seems to be borking things. npm publish works though 😄

[RobGraham]

Also receiving invalid tar file errors when installing dependencies in our project. Each yarn attempt seems to bork in different dependencies while npm commands have no issues

[strarsis]

Same issue with different package:
yarn install v1.3.2 [1/5] Validating package.json... [2/5] Resolving packages... [3/5] Fetching packages... warning Pattern ["css@https://github.com/pocketjoso/css.git"] is trying to unpack in the same destination "/home/build/.cache/yarn/v1/npm-css-2.2.1-8ddea7e3cbc0a183ecf694a7a5fbc84326893893" as pattern ["css@^2.2.0","[email protected]","css@^2.1.0","[email protected]"]. This could result in a non deterministic behavior, skipping. error An unexpected error occurred: "https://github.com/pocketjoso/css.git: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?". info If you think this is a bug, please open a bug report with the information provided in "[...]/yarn-error.log". info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
nvm 0.33.6
node v8.9.3
yarn 1.3.2
Ubuntu 16.04.3 LTS

[serkanbektas]

@strarsis I have just same problem. Could you fix it then?

[strarsis]

@serkanbektas: IIRC I either used a different release with different package dependency version (see https://github.com/reworkcss/css/issues/109 and https://github.com/gulp-sourcemaps/gulp-sourcemaps/issues/334) / installed the packages using npm instead yarn.

[serkanbektas]

@strarsis Which package did you use css.git package with? I try for grunt-critical package. I tried different release, but nothing happened.

[strarsis]

@serkanbektas: yarn add grunt-critical works for me – but this is the latest release.
Of what package grunt-critical is a dependency and what specific version it got?