Feathers: Very slow throughput

[averri]

Steps to reproduce

1. Install autocannon.

2. Create a file fastify.js with the following content (this is for the test baseline, using another framework):

// Require the framework and instantiate it                                      
const fastify = require('fastify/fastify')({ logger: false })                    

// Declare a route                                                               
fastify.post('/users-without-db', async (request, reply) => {                    
    return { hello: 'world' }                                                    
})                                                                               

// Run the server!                                                               
const start = async () => {                                                      
    try {                                                                        
        await fastify.listen(3030)                                               
        fastify.log.info(`server listening on ${fastify.server.address().port}`) 
    } catch (err) {                                                              
        fastify.log.error(err)                                                   
        process.exit(1)                                                          
    }                                                                            
}                                                                                
start()                                                                          

1. Run Nodejs:

node fastify.js

1. Run autocannon agaist the Node service:

[root@server]# autocannon -c 100 -H Content-Type=application/json -i body.json -m POST http://localhost:3030/users-without-db
Running 10s test @ http://localhost:3030/users-without-db
100 connections

┌─────────┬──────┬───────┬───────┬───────┬──────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%   │ 97.5% │ 99%   │ Avg      │ Stdev   │ Max      │
├─────────┼──────┼───────┼───────┼───────┼──────────┼─────────┼──────────┤
│ Latency │ 6 ms │ 10 ms │ 22 ms │ 28 ms │ 10.95 ms │ 4.36 ms │ 71.05 ms │
└─────────┴──────┴───────┴───────┴───────┴──────────┴─────────┴──────────┘
┌───────────┬────────┬────────┬─────────┬─────────┬─────────┬─────────┬────────┐
│ Stat      │ 1%     │ 2.5%   │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min    │
├───────────┼────────┼────────┼─────────┼─────────┼─────────┼─────────┼────────┤
│ Req/Sec   │ 5471   │ 5471   │ 8919    │ 9999    │ 8738.8  │ 1236.16 │ 5470   │
├───────────┼────────┼────────┼─────────┼─────────┼─────────┼─────────┼────────┤
│ Bytes/Sec │ 898 kB │ 898 kB │ 1.46 MB │ 1.64 MB │ 1.43 MB │ 203 kB  │ 897 kB │
└───────────┴────────┴────────┴─────────┴─────────┴─────────┴─────────┴────────┘

Req/Bytes counts sampled once per second.

87k requests in 10.08s, 14.3 MB read

1. Note the average requests per second using that framework: 8738.8 req/s.

2. Stop Node.

3. Create a file feathers.js with the following content:

const feathers = require('@feathersjs/feathers');
const express = require('@feathersjs/express');
const socketio = require('@feathersjs/socketio');
const app = express(feathers());

const appPort = 3030;
app.use(express.json());
app.use(express.urlencoded({extended: true}));
app.configure(express.rest());
app.configure(socketio());
app.use(express.errorHandler());
app.use(express.errorHandler());

class UsersWithoutDB {

    async create(data, params) {
        return data
    }
}

app.use('/users-without-db', new UsersWithoutDB());

app.listen(appPort).on('listening', () => {
    console.log(`Server listening on localhost:${appPort}`);
});

1. Run Nodejs:

node feathers.js

1. Run autocannon agaist the Node service:

[root@server]# autocannon -c 100 -H Content-Type=application/json -i body.json -m POST http://localhost:3030/users-without-db
Running 10s test @ http://localhost:3030/users-without-db
100 connections

┌─────────┬───────┬───────┬───────┬───────┬──────────┬──────────┬───────────┐
│ Stat    │ 2.5%  │ 50%   │ 97.5% │ 99%   │ Avg      │ Stdev    │ Max       │
├─────────┼───────┼───────┼───────┼───────┼──────────┼──────────┼───────────┤
│ Latency │ 22 ms │ 28 ms │ 72 ms │ 89 ms │ 32.02 ms │ 13.06 ms │ 166.81 ms │
└─────────┴───────┴───────┴───────┴───────┴──────────┴──────────┴───────────┘
┌───────────┬────────┬────────┬────────┬─────────┬────────┬────────┬────────┐
│ Stat      │ 1%     │ 2.5%   │ 50%    │ 97.5%   │ Avg    │ Stdev  │ Min    │
├───────────┼────────┼────────┼────────┼─────────┼────────┼────────┼────────┤
│ Req/Sec   │ 1334   │ 1334   │ 3355   │ 3795    │ 3079.7 │ 815.78 │ 1334   │
├───────────┼────────┼────────┼────────┼─────────┼────────┼────────┼────────┤
│ Bytes/Sec │ 370 kB │ 370 kB │ 929 kB │ 1.05 MB │ 853 kB │ 226 kB │ 370 kB │
└───────────┴────────┴────────┴────────┴─────────┴────────┴────────┴────────┘

Req/Bytes counts sampled once per second.

31k requests in 10.09s, 8.53 MB read

1. Notice that the requests per second of the service using Feathers is very low, around 3000 requests per second. Another framework have higher throughput in the same server.

Expected behavior

I believe that the throughput should be higher than 3000 requests per second, I have seen many benchmarks reporting much higher values.

Actual behavior

The throughput is very low.

System configuration

CPU Intel Core i7 8650 2.11 GHz, 4 cores, 32 GB RAM, SSD, Windows 10 64 bits.

Module versions (especially the part that's not working):

@feathersjs/[email protected]
@feathersjs/[email protected]
@feathersjs/[email protected]

NodeJS version: v10.15.3

Operating System: Centos 7

Browser Version: N/A

React Native Version: N/A

Module Loader: require

[daffl]

Since Feathers does not add a lot of overhead, these are probably the same number as for any other Express application. If you are concerned about performance you should use Feathers with websockets which can be 2 to 200 times faster than even Fastify as shown in this comparison.

Feathers v4 will also come with a native HTTP2 adapter which should provide similar (and probably better) numbers.

[averri]

Hi @daffl, thanks for the information. Yes, that's true about websockets, it's really amazing but I need the REST API. Feathers is the best framework in terms of simplicity to create REST APIs.

You are right about the equivalent performance of Express. I have gotten similar numbers. I tested Feathers and Express on a Linux server, VPS. It's also slow! It seems that the only way to get that high numbers (above 30k req/s) is to use a server with very high end specs.

Please let me know if that makes sense.

[viettd]

Hi @averri,
Instead of scale up as you mentioned, I think scale out ( add more servers ) is a better option?

[daffl]

The next version will add the option to use Koa instead of Express as the underlying HTTP framework. It uses the same radix tree router that is already used for Feathers websocket transports and will probably allow a similar throughput to Fastify. The work in progress branch is https://github.com/feathersjs/feathers/tree/koa.

[averri]

@daffl, I have seen benchmarks of Koa outperforming Express by a small margin, do you have a roadmap to share? I'm quite happy with the set of features of Feathers. I think it should continue focusing on providing a transport agnostic API and do it well, as it already does.

I agree with @viettd, scaling out is the way to go, especially because in a real application it's unlikely that a single service will sustain a very high throughput using just one server.

[daffl]

In general, since a Feathers transport adapter does not really add much overhead, it will be about as fast as the underlying framework being used. The new authentication makes it much easier to implement a new transport with Koa being the first. This is one of the nice things about Feathers. You can choose the most suitable available transport without having to change anything in your application itself.

I also think it is worth noting that plain throughput will probably not matter that much as soon as you make a database request or even just add authentication. Almost all performance problems I have ever encountered were inefficient database queries not anything at the HTTP framework level.

[anli-xsigns]

Did Koa support arrive in feathers v4? Is there a working example somewhere, maybe with authentication and rest? Or are there other options than using express?