Deno: Question: install scripts with external assets

[swwind]

deno install is excellent for installing scripts, but it seems that it could only install ts (and js?) files through dependencies.

For example, std/http/file_server.ts is a simple file server which needs some html template to generate page, but it could only put all html template files(and even a lot of css files) inside the ts file as a string. Maybe it works, but it is not elegant and graceful enough to manage these strings. And this solution is not suit for images(such as favicon.ico) as they are binary files.

More about this example you can see this: https://github.com/denoland/deno/blob/master/std/http/file_server.ts#L200-L288
(It puts all the html and css inside the ts file as a string which is ugly)

The best solution I found is to clone the whole repo(with html and css files outside of ts file) to disk and use deno install ./index.ts manually to install my scripts locally. This install method is too complex and not fascinating at all; thus I was think of some feature such as deno install https://github.com/name/repo to run the same as git clone ... && deno install -n repo ./repo/xxx.ts?

Looking forward to your reply and thanks in advance.

[MarkTiedemann]

deno install is excellent for installing scripts, but it seems that it could only install ts (and js?) files through dependencies.

I'd assume that this is for security reasons. Imagine deno install putting an executable into your path.

And this solution is not suit for images(such as favicon.ico) as they are binary files.

While it may not be "suitable", you can store binary data in a TypeScript text file, for example, in a Uint8Array:

import { equal } from "https://deno.land/[email protected]/testing/asserts.ts";

let res = await fetch("https://deno.land/favicon.ico");
let icon_from_url = new Uint8Array(await res.arrayBuffer());
await Deno.writeTextFile("favicon.ts", `export let icon = new Uint8Array([${icon_from_url.toString()}])`);
let icon_from_file = (await import("./favicon.ts")).icon;
equal(icon_from_url, icon_from_file);

Note that this will increase file size (e.g. favicon.ico is 15kb whereas favicon.ts is 34kb).

---

EDIT: To keep the size similar, you can put the binary file into a JS template string:

import { equal } from "https://deno.land/[email protected]/testing/asserts.ts";

function escape(s: string) { return s.replace(/[\\`$]/g, "\\$&").replace(/\r/g, "\\r") }
function unescape(s: string) { return s.replace(/\\`/g, "`").replace(/\\\$/g, "$").replace(/\\r/g, "\r").replace(/\\\\/g, "\\") }

let res = await fetch("https://deno.land/favicon.ico");
let icon_from_url = await res.text();
await Deno.writeTextFile("favicon.ts", `export let icon = \`${escape(icon_from_url)}\``);
let icon_from_file = unescape((await import("./favicon.ts")).icon);
equal(icon_from_url, icon_from_file);

In this case, favicon.ts is only 19kb, just 4kb more than the original file.

(Please note that this code is just the minimum to make it work on my machine. YMMV.)

[nayeemrmn]

Duplicate of #5987, effectively. I'm not sure if deno install is a special case.

[swwind]

@MarkTiedemann Thanks for your help, but what I want to emphasize is bundle those non-code assets into TypeScript file does not make sense to me. Simply I just want a way to import these non-TypeScript files as ArrayBuffer or anything else which can be recognized by deno install.

@nayeemrmn I think that issue is relative to mine, but load non-importable resources though import is not the only possible solution for mine.

For example, we can add a manifest file for deno install, which listed all the files which is required to install the script.
Such as:

// manifest.json
{
  "files": {
    "a": "a.ts",
    "b": "b.png"
  },
  "main": "a"
}

and in a.ts:

const b = Deno.getAssetById("b"); // get Uint8Array or anything else

and install the scripts by deno install ./manifest.json instead.

In conclusion, I think deno install should be more extendable and powerful to install full-apps but not only TS scripts.

[swwind]

As for security, I think deno install is the same as npm install -g so long as you don't install scripts from weird source.

[nayeemrmn]

I think that issue is relative to mine, but load non-importable resources though import is not the only possible solution for mine.

For example, we can add a manifest file for deno install, which listed all the files which is required to install the script.
Such as:

This is also a solution to #5987... the problems are tightly coupled. deno install just creates a shortcut script which invokes the deno run command for you. There's nothing that should only work for deno installed scripts.

[MarkTiedemann]

I just want a way to import these non-TypeScript files as ArrayBuffer

Deno.getAssetById("b"); // get Uint8Array

bundle those non-code assets into TypeScript file does not make sense to me

I don't quite understand what you're saying.

The example code I showed was meant to illustrate how you can build a TypeScript file that contains binary data. So if you have a favicon.ico file, you can build a favicon.ts file that allows access to the binary data.

I don't quite get the difference between your suggested Deno.getAssetById("favicon") and my suggested import { favicon } from "./favicon.ts". Both give you a Uint8Array (or similar).

Granted, my setup would require an additional build step on your part, but I don't feel like that's unreasonable.

[MarkTiedemann]

As for security, I think deno install is the same as npm install -g so long as you don't install scripts from weird source.

No, deno install is a safe command whereas npm install -g with a postinstall hook can add your computer to a crypto-mining farm.

[kitsonk]

npm install -g with a postinstall hook ~can add~ _has added_ your computer to a crypto-mining farm (_in the past_)

[swwind]

Granted, my setup would require an additional build step on your part, but I don't feel like that's unreasonable.

Well, I just want deno to do that stuff for me(I'm lazy). In other words, I do hope some implementation like #5987.

npm install -g with a postinstall hook ~can add~ _has added_ your computer to a crypto-mining farm (_in the past_)

I see, that's horrible, I used to think they just print some ads for me.

Closed as duplicated to #5987.

[MarkTiedemann]

Well, I just want deno to do that stuff for me(I'm lazy). In other words, I do hope some implementation like #5987.

@swwind Would you be fine with a third-party tool doing this for you? I'm thinking about building a tool that would do this for you, i.e. in-line asset files into JS/TS source files.

[swwind]

That's a good idea. I appreciate lightweight tools like this.

[nayeemrmn]

https://github.com/axetroy/deno_pkger

[MarkTiedemann]

@swwind Documentation is missing, but the code is useable and tested: https://github.com/MarkTiedemann/deno_pack. Install with deno install --allow-read --allow-write https://raw.githubusercontent.com/MarkTiedemann/deno_pack/master/pack.ts.