Plumber: Single process per R script/endpoint?

[cjtexas]

I have installed the package and am using the pm2 process manager and as far as I can tell it's just running a single R process no matter how many GET requests I make. Is there any way of spinning up a new R process for each new request?

Thanks!

[cjtexas]

I did try the pm2 -i argument already and pm2 monit says the service is running in fork mode.

[trestletech]

So there are a couple of possible approaches here.

One relies on pm2 and _should_ work, but your comment is perhaps suggesting that it's not working for you. pm2 supports the idea of having a pool of workers supporting a single endpoint and it would load-balance the traffic between them. That should work, but each process would still only be able to handle a single request at a time. (If you're saying that's not working for you, let me know and that will be another prod for me to finally formalize that documentation and get a working demo together).

The other approach would be much more complicated, but would be to emulate the type of request handling that RServe does (http://illposed.net/rserve.html). Basically, it spawns an R process, does the initialization work, and then any time an incoming request arrives, it forks the process and the new child process is the one actually responsible for handling the request. This works well from a performance perspective, but there are some concerns about resource sharing/conflicts, and Windows compatibility.

[trestletech]

@onehundredpercenttexan just to confirm your experience, it looks like pm2's load balancing infrastructure is coupled to NodeJS and isn't likely to work with R or a shell script. :confused:

So option one described above won't be doable with just pm2. It would require starting up a pool of workers in pm2 then configuring nginx or HAproxy in front to do the load balancing. That sounds gross enough to me that I'm interested in another solution.

It might be worth considering using R/httpuv to handle the load balancing, but I'd need to do some more research there before I could say.

[NLKNguyen]

How is the progress on this?
What do you think about leveraging Docker Swarm? Each container, with auto restart policy, has only one R plumber process, and Swarm will automatically do the load balancing. In Docker 2016 keynote 1, they presented that.

[number-cruncher]

I too have this use case -- needing R to handle 1+ API request at a time. Anyone know of an example of how someone has solved for this use case?

[nihonjinrxs]

+1 for this enhancement.

We're currently handling this by running multiple Docker containers, each with a plumber process, behind an ELB on AWS -- seems OK as a workaround, but is much more infrastructure-heavy than I'd like, and only works for a truly stateless API.

[FvD]

I've been playing around with the future package by Henrik Bengston. And it seems to play nice with plumber, meaning that we have a host of options for handling plumber processes in an asynchrounous, multicore or cluster. With future.BatchJobs it can even hook into the BatchJobs.

This would mean that we can solve the original question @onehundredpercenttexan (spin up mutliple R processes) without any change to plumber, and even extend that with the additional options that future offers.

But I may be missing something. Here is a minimal example set up as a multiprocess plan that returns a different pid for each endpoint. It even returns a different pid for different calls to the same endpoint.

library(future)
plan(multiprocess)

#* @get /
api_version <- function(){
  version <- list(Version = "0.0.1", 
                  Comments = "use future to handle api calls asynchronously")
  return(version)
}

#* @get /a
a <- function(){
  process_a %<-% {
       Sys.sleep(2)
       current_pid <- Sys.getpid()
       return(current_pid)
  }
 output <- list("Handled in pid: " = process_a)
 output
 }

#* @get /b
b <- function(){
  process_b %<-% {
       Sys.sleep(2)
       current_pid <- Sys.getpid()
       return(current_pid)
  }
 output <- list("Handled in pid: " = process_b)
 output
}

#* @get /c
c <- function(){
  process_c %<-% {
       Sys.sleep(2)
       current_pid <- Sys.getpid()
       return(current_pid)
  }
 output <- list("Handled in pid: " = process_c)
 output
}

[jaehyeon-kim]

@FvD

I followed the example and this is what I find/speculate.

1. A function is processed in different processes as you point out.
2. However, while # concurrent requests increases, median response time increases almost linearly. I guess the function itself seems to run sequentially.

In this regards, the suggested way isn't necessarily useful to handle multiple concurrent requests. On the other hand, if an output of a function needs to be obtained in parallel, it would be of help.

[number-cruncher]

We also suspect it was linear. We threw hardware at it but eventually moved to a paid solution: Blockspring.com

[ZJUguquan]

@FvD
Hi, I have tried your example code, but I'm sorry that I can not reproduce the result. >﹏<
Whichever page of 127.0.0.1:5000/a or 127.0.0.1:5000/b or 127.0.0.1:5000/c I visit , I always get the same pid.
This is my sessionInfo():

R version 3.4.4 (2018-03-15)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 16299)

Matrix products: default

locale:
[1] LC_COLLATE=Chinese (Simplified)_China.936 
[2] LC_CTYPE=Chinese (Simplified)_China.936   
[3] LC_MONETARY=Chinese (Simplified)_China.936
[4] LC_NUMERIC=C                              
[5] LC_TIME=Chinese (Simplified)_China.936    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] future_1.8.0

loaded via a namespace (and not attached):
[1] compiler_3.4.4   parallel_3.4.4   listenv_0.7.0    codetools_0.2-15
[5] digest_0.6.15    globals_0.11.0  

Any suggestions please ... ?

[FvD]

Hi @ZJUguquan, I'm sorry for replying so late. On my list the answers of @jaehyeon-kim and @number-cruncher show that my approach simply does not work. I've been playing around with how to solve this, especially now that there is an example of how @jcheng5 and is team are using the package promises to include async in Shiny. There is a great intro to this in this video that you might want to look at.

My suggestion is that you look at that RStudio video and the promises package.

[shizidushu]

I find it is possible to use Docker swarm and Traefik to do load balance and reverse proxy. Traefik also makes it possible be stateful with sticky-sessions.(I didn't test this feature). Here is the example of docker-compose.yml files. Hopes it could help someone with the load balance problem.

version: "3.6"
services:
  traefik:
    image: traefik:1.6
    command:
      - "--api"
      - "--entrypoints=Name:http Address::80"
      - "--entrypoints=Name:https Address::443 TLS"
      - "--defaultentrypoints=http,https"
      - "--docker"
      - "--docker.swarmMode"
      - "--docker.domain=localhost"
      - "--docker.watch"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - webgateway
      - traefik
    ports:
      - target: 80
        published: 80
        mode: host
      - target: 443
        published: 443
        mode: host
      - target: 8080
        published: 8080
        mode: host
    deploy:
      mode: global
      placement:
        constraints:
          - node.role == manager
      update_config:
        parallelism: 1
        delay: 10s
      restart_policy:
        condition: on-failure
networks:
  webgateway:
    driver: overlay
    external: true
  traefik:
    driver: overlay

# I add `- "traefik.backend.loadbalancer.sticky=true"` just for reference. 
## I didn't test it. But it should works.
version: '3.6'
services:
  app1:
    image: trestletech/plumber
    command: /app/plumber.R
    volumes:
      - ../app2:/app
    deploy:
      labels:
        - "traefik.port=8000"
        - "traefik.docker.network=webgateway"
        - "traefik.frontend.rule=PathPrefixStrip:/app1"
        - "traefik.backend.loadbalancer.sticky=true"
      replicas: 5
      restart_policy:
        condition: on-failure
    networks:
      - webgateway
  app2:
    image: trestletech/plumber
    command: /app/plumber.R
    volumes:
      - ../app2:/app
    deploy:
      labels:
        - "traefik.port=8000"
        - "traefik.docker.network=webgateway"
        - "traefik.frontend.rule=PathPrefixStrip:/app2"
        - "traefik.backend.loadbalancer.sticky=true"
      replicas: 1
      restart_policy:
        condition: on-failure
    networks:
      - webgateway
networks:
  webgateway:
    driver: overlay
    external: true

[mskyttner]

I have tried using n-1 of my n available cores (n=4 when testing on my laptop) to serve concurrent requests against a plumber API. This setup uses docker and docker-compose (and doesn't require but can use swarm and it uses nginx-proxy instead of traefik) and can be found here:

https://github.com/bioatlas/data-mobilization-pipeline/blob/master/sweref99-to-wgs84/docker-compose.yml#L30-L46

To scale up and stress test, these commands were used:

https://github.com/bioatlas/data-mobilization-pipeline/blob/master/sweref99-to-wgs84/Makefile#L32-L36

Looking at the benchmark numbers reported by ab:

Before make scale-up:

Concurrency Level:      4
Time taken for tests:   12.261 seconds
Complete requests:      400
Failed requests:        0
Total transferred:      20226713 bytes
HTML transferred:       20165513 bytes
Requests per second:    32.62 [#/sec] (mean)
Time per request:       122.610 [ms] (mean)
Time per request:       30.653 [ms] (mean, across all concurrent requests)
Transfer rate:          1611.01 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    45  122  30.8    126     221
Waiting:       23  103  34.2     98     219
Total:         46  122  30.8    126     221

Percentage of the requests served within a certain time (ms)
  50%    126
  66%    139
  75%    144
  80%    149
  90%    156
  95%    178
  98%    202
  99%    210
 100%    221 (longest request)

After make scale-up:

Concurrency Level:      4
Time taken for tests:   6.385 seconds
Complete requests:      400
Failed requests:        0
Total transferred:      20226330 bytes
HTML transferred:       20165130 bytes
Requests per second:    62.65 [#/sec] (mean)
Time per request:       63.846 [ms] (mean)
Time per request:       15.961 [ms] (mean, across all concurrent requests)
Transfer rate:          3093.76 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    24   63  60.4     50     705
Waiting:       22   60  58.9     47     704
Total:         24   63  60.4     50     705

Percentage of the requests served within a certain time (ms)
  50%     50
  66%     64
  75%     77
  80%     85
  90%     98
  95%    121
  98%    164
  99%    415
 100%    705 (longest request)

[schloerke]

Going to close this issue as plumber now handles promises / future natively.