Deno: Idea: Deno Workspace

[KSXGitHub]

Rationale

• We want a way to easily manage dependencies:
  
  
  
  • We want import lines to be shorter, so we need import-map, but we don't want to write them by hand.
  
  
  • We want to easily update our dependency versions without manually search-and-replace.
  
  
• We want to easily grant permissions to scripts:
  
  
  
  • --allow-all is convenient, but risky.
  
  
  • --allow-foo --allow-bar --allow-baz ... is long and hard to type.
  
  

Suggestion

• Implement a deno-workspace file that Deno automatically loads if it exists in current working directory.
• This deno-workspace file...
  
  
  
  • ...is NOT equivalent to NPM's package.json: Unlike package.json, deno-workspace file won't be published to registry, and even if it does, Deno will not load it.
  
  
  • ...is equivalent to tsconfig.json, pnpm-workspace.yaml, stack.yaml, Makefile and other build file formats: Deno and other tools may use it to build binaries.
  
  

Requirements

• There is only one deno-workspace file per workspace.
• deno-workspace files of dependencies (should there be any) are completely ignored.
• There is only one file name and one format for deno-workspace file.
  
  
  
  • It might be tempting to support multiple formats, but that would only introduce complexity.
  
  
• Filename:
  
  
  
  • Basename should be deno-workspace.
  
  
  • Extension can be discuss later.
  
  
• If a standard registry API is specified, Deno Workspace should support it out-of-the-box.

Example CLI Commands

# Generate `deno-workspace` file
deno init

# Generate import map and lock file
deno gen-import-map

# Without arguments, `deno fetch` should look for `deno-workspace` file
deno fetch

# Loads `deno-workspace` configuration for REPL session
deno repl

# If LOCAL_FILE belongs to current working directory (directly or indirectly), Deno should load workspace file before executing the script
deno run LOCAL_FILE

# Generate import map, resolve all import URLs so that library consumer won't reply on our import map, and compile to JavaScript
deno make

# Update all dependencies to latest versions.
deno outdated --update

Example deno-workspace file

# I write this in YAML because it is easier to type. Deno Workspace File's format and name need to be discussed further.
registries:
  std: https://deno.land/std/{pkgname}@{pkgver}/mod.ts
  x: https://deno.land/x/{pkgname}@{pkgver} # BTW, how do I specify versions of third party libraries?
  private: https://mycompany/deno/{pkgname}/{pkgver}/index.js
permissions:
  '**/*.ts':
    read:
      - '{workspace-root}/**/*.txt'
    write:
      - '{script-dirname}/{script-basename}.log'
  '@std/server': # This syntax is similar to NPM namespaces
    net: true
license: MIT

Choosing File Format

JSON

• Pros:
  
  
  
  • Native support
  
  
• Cons:
  
  
  
  • Hard and slow to type
  
  
  • No comments
  
  
  • No multi-line string
  
  
  • No trailing comma → Does not work well with git diff and git merge
  
  
  • Property keys must be quoted
  
  
  • Only support JSON types. JavaScript such as Map and Set are not supported.
  
  

JSON5

• Pros:
  
  
  
  • Parsers are available: npm/json5, crates/json5.
  
  
  • Support comments.
  
  
  • Support trailing comma.
  
  
  • Support unquoted property keys.
  
  
• Cons:
  
  
  
  • No multi-line strings.
  
  
  • Only support JSON types. JavaScript types such as Date, Map and Set are not supported.
  
  

TOML

• Pros:
  
  
  
  • Parsers are available: npm/toml, crates/toml.
  
  
  • Re-implementation (if need-be) is relatively easy.
  
  
  • Support comments.
  
  
  • No needs for comma, no needs for trailing comma.
  
  
  • Support multi-line strings.
  
  
  • Support additional JavaScript types: Date.
  
  
• Cons:
  
  
  
  • Indentation of multi-line strings are preserved as-is. No stripping of indentation.
  
  
  • Poor support for nested objects.
  
  
  • Map and Set are not supported.
  
  

RON (https://github.com/ron-rs/ron)

• Pros:
  
  
  
  • Rust parser is available. It can be compiled to WASM.
  
  
  • Support comments.
  
  
  • Support trailing comma.
  
  
  • Support multi-line strings.
  
  
  • Can support any JavaScript type.
  
  
  • Clear semantic: User is required to write constructor name.
  
  
• Cons:
  
  
  
  • Indentation of multi-line strings are preserved as-is. No stripping of indentation.
  
  
  • No native JavaScript parser. It must be compiled to WASM.
  
  

YAML

• Pros:
  
  
  
  • Parsers are available: npm/js-yaml, npm/yaml, crates/yaml.
  
  
  • Support comments.
  
  
  • No need for trailing comma.
  
  
  • Support multi-line strings.
  
  
  • Strip extraneous indentation of a multi-line string.
  
  
  • Can support any JavaScript type.
  
  
• Cons:
  
  
  
  • Big specification. Expect inconsistent result between different parsers.
  
  
  • Some people dislike significant whitespaces (not me).
  
  

Programmable JavaScript/TypeScript

• Pros:
  
  
  
  • Native support.
  
  
  • Extremely flexible.
  
  
  • Support multi-line strings.
  
  
  • Can support any JavaScript type.
  
  
• Cons:
  
  
  
  • External tools can only read, not write. User must edit the file manually.
  
  
  • Multi-line strings are preserved as-is. User must use complicated technique to strip indentation.
  
  

My Preference

• I prefer YAML the most, RON second, JSON5 third, and then TOML.
• I'm against using JSON file and JavaScript file.

[hayd]

For updating dependencies in a deps.ts or importmap.json see a script I put together:
https://github.com/hayd/deno-udd
(it's working but a little noisy/not interactive atm)

The good thing about deno-workspace idea is you can write it as a third party lib... so I think the first thing to do is to do just that: write a proof of concept... if it works well it's _possible_ it could be brought into deno (I suspect ry will consider it a little too magical), but it would be useful as a third party CLI.

[KSXGitHub]

@hayd There are several limitations with your script:

• It replies on parsing of TypeScript.
• It must extract version range from a URL, which might be error-prone.
• It can only support a limited types of URL.
• And of course, it's magical.

[hayd]

It's actually even worse, it doesn't parse it just uses regex.
Similar to your outline it explicitly defines registries and their versioning scheme... so it could do something like that with custom registries.

The main difference is it's active update, without interfering with deno resolution i.e. rather than relying on npm install (or bundle or whatever) to resolve what to download.

It's way too naive to be magical, but thanks! 😄

[KSXGitHub]

The main difference is it's active update

What do you mean by this? That you don't need to re-run npm install? Well, you may perceive generating import-map as npm install, but the different is generation of import-map is enough for script to run (Deno would automatically fetch missing dependencies)

[hayd]

This is aside from your proposal / this issue...

It's best practice to use deps.ts for that very reason. If deps.ts includes all the dependencies your project needs i.e. all external imports are defined there, then (after fetch deps.ts) there are no missing dependencies to fetch.
However udd supports updating multiple files and running tests to ensure it can/does download and run the updated dependencies.

I guess what I meant by "active update" was, after running, you end up with explicit versions in your code (e.g. deps.ts) rather than a range - and that's the thing that's committed and that deno runs.

Anyway 🤷‍♂

[KSXGitHub]

@hayd I see. So deps.ts is just import-map with extra steps? Anyway, I think the choice of whether to use exact version rather than a range should be up to the user (if the registry allows it).

Regarding deps.ts, I would prefer import-map more: To use deps.ts is to execute every single dependencies listed even ones you don't use.

[IllusionPerdu]

Regarding deps.ts, I would prefer import-map more: To use deps.ts is to execute every single dependencies listed even ones you don't use.

@KSXGitHub I have a question : Why add dependencies if you don't use it ?

[KSXGitHub]

Why add dependencies if you don't use it ?

I do not always make one file. If different files rely on different set of dependencies, then some are ought to be unused.

[eliassjogreen]

I made dimp a month ago for easier management of imports in import map but the problem is that import maps don't work for installing etc.

[KSXGitHub]

but the problem is that import maps don't work for installing etc.

By "installing", you mean deno install does not have an --importmap flag like fetch and run? That won't be a problem with Deno Workspace because it compiles to JavaScript in which all import URLs are resolved.

[eliassjogreen]

Yeah

[KSXGitHub]

Now I see one advantage deps.ts has over import maps: Import maps can only be used in development environment.

[Soremwar]

@KSXGitHub It should work only on development

If you want to distribute your code you must transpile and bundle (eventually compile) it first

[KSXGitHub]

It should work only on development

That was what Deno Workspace intended for.

If you want to distribute your code you must transpile and bundle (eventually compile) it first

Compile, yes, but bundle, not necessary. Deno Workspace can be used to generate a module repository, which can be used by other Deno developers.

[nayeemrmn]

See https://github.com/denoland/deno/issues/2584#issuecomment-550394902, "boilerplate" referring to config/workspace files. In general the goal is to achieve things like this as userland conventions represented in code.

That said, it seems like the suggested workflow here is to write a library accompanied with a workspace file allowing you to use abbreviated import specifiers for dependencies -- and compile all of that into a regular library that would run under Deno now. This can be implemented as a third party tool. No need for Deno to load it or even know about it.

[MarkTiedemann]

TBH, I'm fine with long import statements, updating dependencies manually, and explicitly writing down permissions. Those don't really seem like actual problems to me but something that is good practice, though it may be a tiny bit tedious at times.

Adding a new workspace concept, a new file format and new commands to Deno, however, seems like quite a huge increase in complexity.

I feel like this is something that could be addressed by a third party tool if there's a need for it. Especially at this point in time, with most Deno projects being rather small, I don't really see a need for this to be a core feature of Deno.

[BentoumiTech]

@KSXGitHub I was looking for something similar to what you described when I stumbled upon this issue.

I just released a small tool that handles permissions and scripts in a file in a similar fashion as to what you described https://github.com/BentoumiTech/denox

[sanket143]

How about this, for all of our external dependencies, we can keep all our imports in dep.ts and
import from that in our project files.

It'll be easier to go through all remote dependencies, and their current versions.

_dep.ts_

import { serve } from "https://deno.land/[email protected]/http/server.ts";

export {
  serve
}

_main.ts_

import { serve } from "./dep.ts";

const s = serve({ port: 8000 });

console.log("http://localhost:8000/");

for await (const req of s) {
  req.respond({ body: "Hello World\n" });
}