pnpm as a service

[zkochan]

If pnpm will be running in the background as a service, a pnpm install would just be a call to the service that already runs. It would:

1. decrease startup time
2. remove the need in locking the store and project

The service would also keep the store warm. So it would be possible to make every install in offline mode.

This would be not obligatory. pnpm would work with or without the service. On CI servers, for instance, such service wouldn't make any sense.

[zkochan]

@vjpr, @etamponi, I suggest moving the discussion here.

So the way I was thinking about this. There would be a separate application (a pnpm service)
that would run in the background. pnpm would then check if there's this service running
and if it's there, then the requests for getting the packages would be done by the service.
Also, all the store-manipulation things would be done by the service.

One service would be able to control one store.

[vjpr]

There would be a separate application (a pnpm service) that would run in the background.

I think the Flowtype cli is a good example of a similar daemonised tool that we can learn from.

I would prefer it to be seamlessly built into pnpm. So you could pass a flag to use it and it will create a service if it doesn't exist. I really don't want to have to manage a separate service.

---

Some discussion points on the usage (inspired by flowtype):

How to use the daemon?

// Starts the deamon if not started.
pnpm install --use-daemon/daemon
// Doesn't start a daemon if not already running. Throws error.
pnpm install --daemon --no-daemon-auto-start

// Namespaced for pnpm to avoid conflicts with npm.
pnpm config set pnpm-use-daemon

How to control the daemon?

Running in background:

pnpm daemon/server start/stop/restart/status --background

Running in foreground:

pnpm daemon/server foreground/fg

[zkochan]

I there some cross-platform library that can start a Node process in the background and keep it running?

[zkochan]

This one seems good: https://github.com/IonicaBizau/diable

[etamponi]

I feel that, for simplicity sake, having an option to start pnpm in "daemon
mode" and then other pnpm just connect to that one daemon is a better
solution, at least initially. I say this because we have tons of ways of
managing services: pm, systemd, runit, you name it! Why adding yet another
way of starting a daemon? And in a way that is really difficult to control:
if a pnpm install process also starts a daemon that then other pnpm install processes can connect to, the burden of handling lifetime, error
handling, restarting, message passing, is left to pnpm! But why, I say?
It's just one systemd unit away, or if you don't like systemd, you just
start pnpm in daemon mode in the background using 3 lines of bash.

I feel that if we take the path of supporting a self-spawning daemon as an
option, it would only give pnpm problems and weird side conditions to
handle.

My solution: let's keep it simple, at least at first:

1) pnpm install keeps working normally, eventually connecting to an
already running daemon, but it won't start its own daemon
2) pnpm daemon would start the store-managing daemon, and only this (in
the foreground, and with an eventual (and in my opinion redundant)
--background switch.

Sorry if I sound aggressive: I'm trying to be as clear as possible, it's
not my intention to attack anyone! :)

Il 08 dic 2017 23:10, "Zoltan Kochan" [e-mail] ha scritto:

This one seems good: https://github.com/IonicaBizau/diable

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-350384822, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNIHIcT7DvQTMsO51ji5JKxihlAVNks5s-bPRgaJpZM4OJEUt
.

[zkochan]

haha, relax. I am from Eastern Europe, not America;-) That wasn't aggressive at all, you can express yourself the way you like.

Regarding the solution, I am totally fine having the dedicated command. Easier is always better in my opinion. Lets' do the background flag in a future version if this will become successful.

[vjpr]

@etamponi @zkochan The daemon/service should be entirely controlled by pnpm. Just like flow. The user should never deal with systemd, etc. I always find linux services painful to deal with. (E.g. Why didn't it start, where are the logs, permissions issues, hard-wired init.d flags in some apt packages, etc, etc.)

Also, pnpm version changes a lot, I would like the service to automatically know when it needs to restart after a pnpm install -g pnpm.

I would also use pnpm server rather than pnpm daemon. Daemon makes me think of linux services, and having something running all the time.

[vjpr]

@zkochan How would you communicate between processes? https://www.npmjs.com/package/node-ipc?

[etamponi]

http is better I think. Please, if possible, don't use ipc because it would
break my usecase... I would run pnpm install inside docker, and it has to
communicate to the server that has to run on the host. So ipc won't work
for me.

Http is also easier to understand.

As for the daemon/server debate: I think you're underestimating the amount
of complexity that you're asking to add. Spawning a background job and
maintaining it running is not an easy task, that's why linux daemons are so
complex.

As for pnpm daemon vs pnpm server, I don't care... It's just a name
change, right? The working flow wouldn't change

Il 09 dic 2017 09:31, "Vaughan Rouesnel" [e-mail] ha
scritto:

@zkochan https://github.com/zkochan How would you communicate between
processes? https://www.npmjs.com/package/node-ipc?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-350432767, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNKlKTk0U_0_X_NVu9PBAVsC2e7Bpks5s-kVkgaJpZM4OJEUt
.

[vjpr]

I would run pnpm install inside docker, and it has to
communicate to the server that has to run on the host.

@zkochan Does @etamponi's docker use case work or would it involve unsupported cross-device hard linking?

[etamponi]

I can already run pnpm inside my containers and it works flawlessly.

Il 09 dic 2017 09:41, "Vaughan Rouesnel" [e-mail] ha
scritto:

I would run pnpm install inside docker, and it has to
communicate to the server that has to run on the host.

@zkochan https://github.com/zkochan Does @etamponi
https://github.com/etamponi's docker use work or would it involve
unsupported cross-device hard linking?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-350433147, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNCe1sQbIl55dH2zGOWVj_eA-lg5Oks5s-ke4gaJpZM4OJEUt
.

[etamponi]

Of course I mean that I can run it with a shared pnpm store, and I've
already managed to get rid of the cross-device hard linking errors :)

Il 09 dic 2017 09:42, "Emanuele Tamponi" [e-mail] ha
scritto:

I can already run pnpm inside my containers and it works flawlessly.

Il 09 dic 2017 09:41, "Vaughan Rouesnel" [e-mail] ha
scritto:

I would run pnpm install inside docker, and it has to
communicate to the server that has to run on the host.

@zkochan https://github.com/zkochan Does @etamponi
https://github.com/etamponi's docker use work or would it involve
unsupported cross-device hard linking?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-350433147, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNCe1sQbIl55dH2zGOWVj_eA-lg5Oks5s-ke4gaJpZM4OJEUt
.

[vjpr]

I've already managed to get rid of the cross-device hard linking errors :)

How? You fallback to copying?

[zkochan]

Clearly from DX perspective, running it automatically and in the background is much preferred than running in the foreground. Lets' have a separate issue created for the background option, so that we can concentrate on one issue at a time. Issue for backgroud service: #960

Is communicating via IPC faster than via HTTP? We can start with HTTP but if IPC is faster, we'll have to make it configurable and IPC as default.

[etamponi]

@vjpr falling back to copying would of course remove all the space and speed benefits of using pnpm and the shared store, so no, I don't fallback to copying. It works the way it's intended to :) I just bind-mount the store and override the store config variable to point there.

@zkochan http might be slower than memory-shared ipc for intense communication between processes. As far as I can see, in this case the client and server will need to send just a few very small messages, making the tradeoff between http and ipc hard to evaluate. Let's consider that the main latency will be outside of the communication channel. On the other hand, localhost-to-localhost http communication is in the sub-millisecond speed order: if we assume an average of 10 to 100 messages exchanged between the client and the server, the communication channel always adds less than 100ms to the installation time.

EDIT: I also agree that from DX prospective, running the server automatically and in the background is better -- I am just worried that it adds complexity, so I completely support splitting that, I am not against implementing it in the future :)

[etamponi]

@zkochan just checked out node-ipc, it looks like it uses tcp/udp/unix socket connections. All of them can be used to communicate between a Docker container and a process running outside of it, so you can also use node-ipc directly, as far as I am concerned :) I was worried they'd use some kind of OS-specific connection.

[etamponi]

Yet another suggestion regarding inter-process communication: I'd say that using a very well-known dependency is better than using a more specific but less well known one. While I was developing a custom Docker volume plugin in Node, I found that the builtin http server + express + unix socket was a perfect match, I could set it up without any problems (you just specify the path to the unix socket instead of the ip/port on which to listen).

node-ipc suggests unix sockets as the fastest mean of communication between local processes, but as you can use unix sockets using a plain node module + express, perhaps it would be good to consider using them instead of node-ipc.

[vjpr]

Re: Docker

pnpm server running on host, pnpm i running in container could end up with two daemons operating on same store. Need to check for this. E.g. print the hostname and process id of the daemon pnpm is using each run, and when there is already one running, do the same.

[etamponi]

This is one of the side cases I was mentioning earlier, but if we avoid
running the daemon automatically when running pnpm install, this issue
will never happen. If I am understanding correctly, we are considering
moving the part that starts the daemon automatically with pnpm install to
a separate issue #960

Il 09 dic 2017 10:51, "Vaughan Rouesnel" [e-mail] ha
scritto:

Re: Docker

pnpm server running on host, pnpm i running in container could end up
with two daemons operating on same store. Need to check for this. E.g.
print the hostname and process id of the daemon pnpm is using each run, and
when there is already one running, do the same.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-350441184, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNL4qCruAQV-Ld-yYjLnSy2EXpokuks5s-lgagaJpZM4OJEUt
.

[vjpr]

if we avoid running the daemon automatically when running pnpm install, this issue will never happen.

By default, pnpm should not run the daemon. It should only run it when its enabled in config or flag.

pnpm install --use-server

Then you can also pass in an option pnpm --use-server install --no-auto-start, to prevent it autostarting.

Given that the docker case with host-based server won't be the most common.

[etamponi]

You're right!

I guess that as long as I can also start the pnpm server independently
and in daemon mode, so that then other pnpm install processes can talk to
it, I am fine with everything else :+1:

Il 09 dic 2017 14:02, "Vaughan Rouesnel" [e-mail] ha
scritto:

if we avoid running the daemon automatically when running pnpm install,
this issue will never happen.

By default, pnpm should not run the daemon. It should only run it when its
enabled in config or flag.

pnpm install --use-server

Then you can also pass in an option pnpm --use-server install
--no-auto-start, to prevent it autostarting.

Given that the docker case with host-based server is an edge case.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-350462467, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNE0rXO09KEXJNEamPFKduBrqW1eqks5s-oTPgaJpZM4OJEUt
.

[zkochan]

I think the interface should be a bit different. If a server runs for a specific store, then it is locked by it. So pnpm can check the lock and the lock can contain the server's port. If it finds the lock with the port number, then it can communicate to the server.

[etamponi]

I am ok, but please also include the ip over which pnpm should communicate
with the server, as pnpm inside the container can't talk over localhost, it
had to use a bridge interface.

Il 09 dic 2017 14:16, "Zoltan Kochan" [e-mail] ha scritto:

I think the interface should be a bit different. If a server runs for a
specific store, then it is locked by it. So pnpm can check the lock and the
lock can contain the server's port. If it finds the lock with the port
number, then it can communicate to the server.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-350465291, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNPoolfMKzs55ALEV0HXLYmdwCk39ks5s-ohJgaJpZM4OJEUt
.

[zkochan]

I might be ready with a PoC this weekend

[zkochan]

This is still in progress. Not as easy as I thought, but doable.

For detecting whether a service runs for a store I'll use a server.json file. If a service runs for the store, the file will exist, with the connection info of the server: hostname/port

[etamponi]

I know I am asking a very specific thing which might not be possible to do, but...

since install hooks are run in the node_modules directory of projects, it should be possible for them to be run by the "client" in the client/server mode of pnpm. Specifically, I would really like that postInstall, defined in pnpm/supi:/src/install/postInstall.ts, would run on the client and not in the store-controller (that is, the server).

I am asking this because of my usecase. On Glitch, the client would run inside a container, so if I can run the install hooks from the client, I've all the security guarantees provided by Docker, and no rogue scripts can mess with the store.

It would save me tons of work, so if it is easy enough for you to do, please do! :)

[zkochan]

There is no reason to move that part to storeController. postInstall is and will be running on the client, not in the store

[etamponi]

Ok cool :) just to know, the links from the store to the project node_modules will be made by the store controller or by the client?

[zkochan]

The client.

On Fri, Dec 15, 2017, 02:07 Emanuele Tamponi [e-mail]
wrote:

Ok cool :) just to know, the links from the store to the project
node_modules will be made by the store controller or by the client?

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-351874285, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB1pm_etcThAyM4rCZlJaI-U3-DC0OH-ks5tAbgxgaJpZM4OJEUt
.

>

Zoltan

[zkochan]

I published an experimental version with 1.24.0-0. The command is called pnpm server. (A better name might be pnpm store server, as we currently scope the store commands under pnpm store.)

So you just need to run pnpm server and then while it runs, the pnpm commands will access the store via this service.

[etamponi]

Great!!! Thank you :) I'll try it tomorrow or Monday! Does it already
support concurrent installs, or they'll still happen sequentially?

Il 16 dic 2017 22:04, "Zoltan Kochan" [e-mail] ha scritto:

I published an experimental version with 1.24.0-0. The command is called pnpm
server. (A better name might be pnpm store server. As we currently scope
the store commands under pnpm store.)

So you just need to run pnpm server and then while it runs, the pnpm
commands will access the store via this service.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-352211727, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNLPhwaF1dlHBnOryGtX6rof8DqZXks5tBDB2gaJpZM4OJEUt
.

[zkochan]

yes, it supports concurrent installs.

[etamponi]

Hooray! Thanks again :)

Il 16 dic 2017 22:11, "Zoltan Kochan" [e-mail] ha scritto:

yes, it supports concurrent installs.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-352212137, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNEoOlK4i-A88K7UD6NtT9xhqouCeks5tBDIbgaJpZM4OJEUt
.

[zkochan]

It is buggy and in beta, so I might publish a few fixes till Monday. Just install the next version of pnpm (pnpm i -g pnpm@next)

[etamponi]

Just tested it... it works great!!! I've a minor request: since I'd like to run pnpm as a different user from the one that runs the store controller, can you add an option to set the permissions on the unix socket and on the _locks directory? Otherwise, setting them to 777 would work too.

[etamponi]

Nevermind, I've just started the server as the same user that then uses the client :) I'll play with it for a while!

[zkochan]

I have issues locally, seems like messages are broken when installing on
big projects. I don't know how that's possible

On Sun, Dec 17, 2017, 13:39 Emanuele Tamponi [e-mail]
wrote:

Nevermind, I've just started the server as the same user that then uses
the client :) I'll play with it for a while!

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-352249568, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB1pm472Z6ZULzNFVsLCFlhYcjVIOFajks5tBP1hgaJpZM4OJEUt
.

>

Zoltan

[zkochan]

I used json-socket for communication via IPC and the sent messages are
sometimes broken json

On Sun, Dec 17, 2017, 14:19 Zoltan Kochan [e-mail] wrote:

I have issues locally, seems like messages are broken when installing on
big projects. I don't know how that's possible

On Sun, Dec 17, 2017, 13:39 Emanuele Tamponi [e-mail]
wrote:

Nevermind, I've just started the server as the same user that then uses
the client :) I'll play with it for a while!

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-352249568, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB1pm472Z6ZULzNFVsLCFlhYcjVIOFajks5tBP1hgaJpZM4OJEUt
.

>

Zoltan

>

Zoltan

[etamponi]

If you tell me on which packages it happens I can help. First things to
check would be if json-socket has default timeouts and message size limits.
Switching to a native http server over the same unix socket would probably
improve reliability too.

Il 17 dic 2017 13:25, "Zoltan Kochan" [e-mail] ha scritto:

I used json-socket for communication via IPC and the sent messages are
sometimes broken json

On Sun, Dec 17, 2017, 14:19 Zoltan Kochan [e-mail] wrote:

I have issues locally, seems like messages are broken when installing on
big projects. I don't know how that's possible

On Sun, Dec 17, 2017, 13:39 Emanuele Tamponi [e-mail]
wrote:

Nevermind, I've just started the server as the same user that then uses
the client :) I'll play with it for a while!

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-352249568, or
mute
the thread
AB1pm472Z6ZULzNFVsLCFlhYcjVIOFajks5tBP1hgaJpZM4OJEUt>
.

>

Zoltan

>

Zoltan

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-352252169, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNK5J5_jahHgbg7Tmv9vo9L_OOaCGks5tBQgwgaJpZM4OJEUt
.

[etamponi]

I meant the built-in node http server

Il 17 dic 2017 13:32, "Emanuele Tamponi" [e-mail] ha
scritto:

If you tell me on which packages it happens I can help. First things to
check would be if json-socket has default timeouts and message size limits.
Switching to a native http server over the same unix socket would probably
improve reliability too.

Il 17 dic 2017 13:25, "Zoltan Kochan" [e-mail] ha
scritto:

I used json-socket for communication via IPC and the sent messages are
sometimes broken json

On Sun, Dec 17, 2017, 14:19 Zoltan Kochan [e-mail] wrote:

I have issues locally, seems like messages are broken when installing on
big projects. I don't know how that's possible

On Sun, Dec 17, 2017, 13:39 Emanuele Tamponi [e-mail]
wrote:

Nevermind, I've just started the server as the same user that then uses
the client :) I'll play with it for a while!

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-352249568, or
mute
the thread
6ZULzNFVsLCFlhYcjVIOFajks5tBP1hgaJpZM4OJEUt>
.

>

Zoltan

>

Zoltan

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-352252169, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNK5J5_jahHgbg7Tmv9vo9L_OOaCGks5tBQgwgaJpZM4OJEUt
.

[zkochan]

It fails when installing dependencies of supi and pnpm. I'd like to keep IPC because it is faster than http (IPC doesn't go through the network card). I will try to add some retry logic. I don't see any message limit or timeout in json-socket

[zkochan]

Retrying seems to help. The retry logic should probably go to the json-socket lib, or a layer between json-socket and @pnpm/server. Also, I wonder if protobuf is faster than JSON.stringify => string encoder

[etamponi]

http can go through an unix socket

Il 17 dic 2017 14:31, "Zoltan Kochan" [e-mail] ha scritto:

Retying seems to help. The retry logic should probably go to the
json-socket lib, or a layer between json-socket and @pnpm/server. Also, I
wonder if protobuf is faster than JSON.stringify => string encoder

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pnpm/pnpm/issues/832#issuecomment-352255876, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjUNPSqm0o0K3BWq4KGOXECP0B8lp2jks5tBReygaJpZM4OJEUt
.

[etamponi]

I was out so I couldn't elaborate more on http, let's try now. I agree with you that using an unix socket is better than going through the network, but let's consider a couple things:

• localhost "network" traffic doesn't go through the network card, it's basically IPC through a tcp-like interface.
• anyway, http can go through an unix socket just fine. This is what, for example, Docker does.

Advantages of using the built-in node http server (require("http")), or a library like express:

• It is _way_ more battle tested than a random library like jsonsocket
• It supports protobuf, e.g.: https://www.npmjs.com/package/express-protobuf
• It is not an unknown library, meaning that it's easier to debug and contribute to
• I can guarantee you that if you go through an unix socket or localhost, you won't see any significant slowdown vs jsonsocket

Anyway, I am willing to take on this part and create a PR that replaces jsonsocket with, for example, express. You can then make any performance test you want, then you can pick the one you prefer :)

[etamponi]

Ok, found an interesting issue: if I specify an invalid/unexisting version in package.json, the store server blocks forever, I've to manually restart it.

My feeling is -- you might say: _again?_ -- that this is probably tied to the complexity of the server code caused by using json-socket. Single routes through an http server would have way less complexity, can handle error states meaningfully, and wouldn't depend on a long-running state. If you don't like single http requests, you can always switch to a websocket and use libraries like socket.io, which handle the socket state for you and have plenty of error handling code built in.

[zkochan]

@etamponi, do the rewrite if you want. I will switch to some other issues in the meanwhile.

The code that works with the json-socket is in the @pnpm/server repo.

I hope you are right and it avoids the network card because in the node-ipc repo it says that only the Unix/Windows Socket avoids the network card

[zkochan]

if you want a high-level framework, better use fastify instead of express, as fastify is faster

[etamponi]

Alright, I'll play around :) Should I only change @pnpm/server? Isn't there any other repo that needs to be changed?

To make sure it uses IPC, I'll use fastify through an unix socket :) any idea about how to solve that error with invalid dependencies in package.json? Should it be enough if I wrap this into a try-catch?

const packageResponse = await ctx.store.requestPackage(wantedDependency, options) // TODO: If this fails, also return the error

[zkochan]

I think you only need to change @pnpm/server. I did all the necessary changes in other repos.

I did not do the error handling, so I guess try/catch and sending some message with the error info.

I was thinking about changing the packageResponse format as it currently contains 3 promises and other fields. We have to respond with those other fields, so maybe they have to be namespaced into body.

https://github.com/pnpm/package-requester/blob/master/src/packageRequester.ts#L43-L62

[zkochan]

@pgonzal, @nickpape-msft I know this is a long thread but you might be interested in this feature. This will make pnpm really fast for monorepos, so rush with pnpm will be a lot more performant.

[etamponi]

https://github.com/pnpm/server/pull/3 makes @pnpm/server use an http server instead of json-socket, and now I can run pnpm install fine on both supi and pnpm :) I found that the issue that json-socket had is probably fixable by adding a backoff path, just as I did with this PR.

[etamponi]

https://github.com/etamponi/pnpm/commit/53bfe88b5f2c163549eeb42daf19dc115e1c87d4#diff-20bf08ec319bd88f8fc9a35b3e0ca98e

This commit contains the changes needed to support the new initOpts format for @pnpm/server.

[zkochan]

ok, I'll merge and publish it today

[octogonz]

@pgonzal, @nickpape-msft I know this is a long thread but you might be interested in this feature. This will make pnpm really fast for monorepos, so rush with pnpm will be a lot more performant.

Interesting. We've discussed many times getting the TypeScript compiler to run as a service (since spinning up the compiler is a significant chunk of our build times). We didn't consider running the installs as a service however...

Can this service perform multiple installs concurrently? For example, our PR builds happen on a lab machine that can be running up to 4 installs in parallel.

[zkochan]

Yes! And the rewrite allowed to implement even installation in several projects simultaneously by a single process: https://github.com/pnpm/pnpm/pull/969

[octogonz]

Awesome! I'll open an issue and see if we can get rush to support this feature.

[zkochan]

@etamponi, your improvements were published in 1.24.0-1 (dist-tag alpha)

[etamponi]

Thank you :)