Codesandbox-client: Add dependency via tar.gz upload or URL

[brownbl1]

🌈 Feature

Sometimes it is nice to provide a demo of a package before it has been published. Or sometimes the package is published in a private npm repository.

Providing the option to reference a dependency in package.json either by uploading the result of npm pack or alternatively specifying an accessible url to that file would make this possible.

[lbogdan]

Hey @brownbl1 ,

Did you try it and it didn't work? I'm asking because our packager uses yarn, and in theory it should support all of the following:

• yarn add <git remote url> installs a package from a remote git repository.
• yarn add <git remote url>#<branch/commit/tag> installs a package from a remote git repository at specific git branch, git commit or git tag.
• yarn add https://my-project.org/package.tgz installs a package from a remote gzipped tarball.

(see yarn add docs for more details)

[brownbl1]

I did try that.

It works locally, but when I add this line to my package.json in codesandbox:
"my-package": "https://my-package.blob.core.windows.net/test/my-package-0.3.2.tgz"

I get the following error:

Could not fetch dependencies, please try again in a couple seconds: Something went wrong while packaging the dependency [email protected]: Command failed: mkdir -p /tmp/264349292 && cd /tmp/264349292 && HOME=/tmp node /var/task/node_modules/yarn/lib/cli add [email protected] --ignore-scripts --no-lockfile --non-interactive --no-bin-links --ignore-engines --skip-integrity-check
error An unexpected error occurred: "https://registry.yarnpkg.com/my-package: Not found".

[layerdev]

hello, please fix this, I can't add dependencies from git URL

[wanderbaby]

I also have this problem. I can't add dependencies from url
"my-package": "https://my-package.blob.core.windows.net/test/my-package-0.3.2.tgz"

[lbogdan]

@brownbl1 @layerdev @wanderbaby Can you please provide a sandbox URL experiencing this behavior?

[hubgit]

Here's an example sandbox: https://codesandbox.io/embed/84wzrj2lv0

It's trying to load this tgz dependency from a URL, and failing:

Could not fetch dependencies, please try again in a couple seconds: Something went wrong while packaging the dependency how-to-embed-a-notebook-in-a-react-app@https://api.observablehq.com/@401.0.0: Command failed: mkdir -p /tmp/2902585221 && cd /tmp/2902585221 && HOME=/tmp node /var/task/node_modules/yarn/lib/cli add how-to-embed-a-notebook-in-a-react-app@https://api.observablehq.com/@401.0.0 --ignore-scripts --no-lockfile --non-interactive --no-bin-links --ignore-engines --skip-integrity-check
error An unexpected error occurred: "https://api.observablehq.com/@401.0.0: Request failed \"404 Not Found\"".

[eps1lon]

Would be a very nice feature to have to play with builds from PRs via CI artifacts.

Did some debugging and the issue seems to be with the packager API. The request url looks ok fine:

https://xi5p9f7czk.execute-api.eu-west-1.amazonaws.com/dev/packages/%40babel%2Fruntime%407.3.1+csbbust%401.0.0+foo%40https%3A%2F%2Fapi.observablehq.com%2F%40jashkenas%2Fhow-to-embed-a-notebook-in-a-react-app.tgz%3Fkey%3D188f50f6a5f19b89+react%4016.8.6+react-dom%4016.8.6

but on the server it gets translated into

mkdir -p /tmp/602449112 && cd /tmp/602449112 && HOME=/tmp node /var/task/node_modules/yarn/lib/cli add foo@https://api.observablehq.com/@426.0.0 --ignore-scripts --no-lockfile --non-interactive --no-bin-links --ignore-engines --skip-integrity-check

for which I don't know how to add tests.

It seems like it is solely an issue with special characters (probably @). For example https://registry.npmjs.org/react-select/-/react-select-3.0.4.tgz is working perfectly fine

[eps1lon]

Though it doesn't work with versions that are not on the registry. In the end the packager will fallback to the version entry it found in the package.json:

https://github.com/codesandbox/dependency-packager/blob/b215a5776ce0251cc6ff0b6a0411a2850251953b/functions/packager/dependencies/install-dependencies.ts#L10

[eps1lon]

While being able to use packages from arbitrary URLs would still be nice the use case of this issue should be solved with https://twitter.com/codesandbox/status/1181184975460282368

[raunofreiberg]

While being able to use packages from arbitrary URLs would still be nice the use case of this issue should be solved with https://twitter.com/codesandbox/status/1181184975460282368

Regarding this CI feature, is it now also possible to use tar.gz URLs directly in package.json on Codesandbox?

[catchmareck]

Has it been solved? When I try to install an external tgz file, ex:

"dependencies": {
    ...
    "angular-bootstrap-md": "https://example.com/angular-bootstrap-md-8.3.1.tgz"
    ...
}

I get the following error message:

Error: Could not fetch https://cdn.jsdelivr.net/gh/https://example.com/angular-bootstrap-md-8.3.1.tgz/package.json

It works fine in a local NPM project.

[chrisdhanaraj]

@lbogdan - hey, just another adding on to the pile. Here's a codesandbox where I've uploaded a tgz and am trying to use it as a local file.

https://codesandbox.io/s/lucid-gates-vh7jn?file=/package.json

[CodeSandboxTeam]

Hello Chris!

This is currently not supported, but you should look out for this in the near future

Thanks for sending the email, it made sure we brought this into our planning for the next steps of Codesandbox.

Best regards
Christian

--- original message ---
On June 5, 2020, 10:34 PM GMT+2 [email protected] wrote:

@lbogdan - hey, just another adding on to the pile. Here's a codesandbox where I've uploaded a tgz and am trying to use it as a local file.

https://codesandbox.io/s/lucid-gates-vh7jn?file=/package.json

—

You are receiving this because you are subscribed to this thread.

Reply to this email directly, view it on GitHub, or unsubscribe.
--- end of original message ---

[chrisdhanaraj]

@CodeSandboxNotifications - any chance for a general ETA?

[christianalfoni]

It is difficult for us to estimate without exploring it a bit more. We have created a POC to unpack .tgz files in the browser, which seems to work. We are working on private packages, but this was not part of the original planning, meaning it is a "nice to have" currently. Hopefully we have estimated the other things well and we get a chance to include this within a few weeks 😄

[CompuIves]

Hey! I have a PR up that adds .tar.gz support to our bundler. I would appreciate it if anyone could try it, also I'm curious whether this solution works for you. I can imagine that the CORS headers are not there, and I'm wondering if it's possible to add them. Otherwise we'll set up a CORS proxy that would allow a URL without CORS headers.

[CompuIves]

This PR has now been merged! You can now either use a tar by uploading it to the sandbox and using file: or by directly referencing a URL in the version field.

[catchmareck]

That's so awesome! Love this feature ❗ 🚀 🎉