Nest: Provide platform-independant way to access underlying http framework

[mojo2012]

Feature Request

Hi folks,

problem

I'm a java dev coming from the Spring universe and just recently started to tinker around with typescript and nestJS (because it is quite similar to Spring). So far I really like it.
What bugs me a bit though is that there is no clear platform-independent way of accessing the underlying HTTP request/response objects.
On various points the nestJS directly exposes the Express or Fastify objects, which makes it hard to switch from one to another.
For example I tried to implement a generic response entity to make it easier to dynamically manipulate the http status, headers and body from the controller:

// controller
@Post("/users")
public async createUser(@Body() body: User): Promise<ResponseEntity<User>> {
    const user = await this.userService.createUser(body)
    return ResponseEntity.of({
        status: HttpStatus.CREATED,
        headers: {
            "Content-Type": "application/json"
        },
        body: user
    })
}

// interceptor to convert this generic response entity to a real response
@Injectable()
export class ResponseEntityInterceptor implements NestInterceptor<ResponseEntity<unknown>, unknown> {
    public intercept(context: ExecutionContext, next: CallHandler<ResponseEntity<unknown>>): Observable<unknown> {
        return next //
            .handle() //
            .pipe(map((responseEntity: ResponseEntity<unknown>) => this.process(context, responseEntity)))
    }

    private process(context: ExecutionContext, responseEntity: ResponseEntity<unknown>): unknown {
        if (responseEntity instanceof ResponseEntity) {
            const http = context.switchToHttp()
            const response: HttpResponse = HttpResponse.of(http.getResponse())

            if (responseEntity.status) {
                response.setStatus(responseEntity.status)
            }

            for (const key in responseEntity.headers) {
                response.setHeader(key, responseEntity.headers[key])
            }

            return responseEntity.body
        } else {
            return responseEntity
        }
    }
}

I implemented a very basic wrapper for the underlying response object:

export class HttpResponse {
    /**
     * The raw HTTP response stream object
     */
    private rawResponse: any

    private constructor(rawResponse: any) {
        this.rawResponse = rawResponse
    }

    public setHeader(header: string, value: string): void {
        if (this.rawResponse.setHeader) {
            // express
            this.rawResponse.setHeader(header, value)
        } else if (this.rawResponse.raw && this.rawResponse.raw.setHeader) {
            // fastify
            this.rawResponse.raw.setHeader(header, value)
        } else {
            throw new Error("Cannot set header: unknown response implementation")
        }
    }

    public setBody(body: unknown): void {
        if (this.rawResponse.raw.json) {
            // express
            this.rawResponse.raw.json(body)
        } else if (this.rawResponse.send) {
            // fastify
            this.rawResponse.send(body)
        }
    }

    /**
     * Sets the HTTP status code
     * @param status the HTTP status code
     */
    public setStatus(status: number): void {
        if (this.rawResponse.status) {
            this.rawResponse.status(status)
        } else {
            throw new Error("Cannot set status: unknown response implementation")
        }
    }

    public static of(rawResponse: any): HttpResponse {
        return new HttpResponse(rawResponse)
    }
}

As you can see I had to add various conditions to check for either fastify or express. This should not be the case, imho.
Maybe there's a better nestJS way for this but I searched to the docs and didn't find any. As at some point there are even express annotations used I figured that's just how it is.

Proposal

It would be really cool to remove the tight coupling to the underlying HTTP frameworks! Furthermore I'd like to see a generic response entity to manipulate HTTP status, headers and body.

Code example

If you are interested in my code: https://github.com/mojo2012/nestjs-demo

[smolinari]

which makes it hard to switch from one to another.

Why would you want to do this (in a single app)?

Scott

[mojo2012]

Why not? No, seriously, if I'm going for a framework like NestJS I'm obviously not interested in the underlying technology. But at some point I might run into problems related to express. I would then like to try fastify without changing all of my code that is bound to express.
Or, at some point a new HTTP server arrives, better than express and fastify together. It is implemented and readily available in nestjs, but I can't use it because I would have to rewrite my express-bound code.
Or, express changes its API and I have to adapt, though I primarily use nestjs.
Or I want to write another framework wrapping/using nestjs.

I could probably think of even more reasons given some time.

[smolinari]

if I'm going for a framework like NestJS I'm obviously not interested in the underlying technology.

And that's just it. You should be interested in the underlying tech as they offer a ton of additional plug-ins/ middleware. Nest builds on top of all of that ecosystem or rather gives you a better environment for larger application built around the underlying tech (and its ecosystem). You shouldn't just think about the request and response objects, but rather you need to see the whole ecosystem of each underlying HTTP framework.

I, for instance, am working with apollo-server-express with @nestjs/graphql. After looking at Fastify and v3 basically being broken with apollo-server-fastify, I have to stick to Express (for now). And, using Express and Apollo Server's Express plugin requires that Nest not mess with Express' request and response objects.

As the NestJS docs say:

Fastify provides a good alternative framework for Nest because it solves design issues in a similar manner to Express. However, fastify is much faster than Express, achieving almost two times better benchmarks results. A fair question is why does Nest use Express as the default HTTP provider? The reason is that Express is widely-used, well-known, and has an enormous set of compatible middleware, which is available to Nest users out-of-the-box.

Scott

[mojo2012]

And that's just it. You should be interested in the underlying tech

I'm interested in a technical way, but not in a "business way"

not mess with Express' request and response objects.

I'm not suggesting to do that. Express would not even know about that nor would any Express plugin/middleware etc.
But if you write a "nest based" plugin you could rely on the (stable) wrapped nest request/response (and maybe more) objects that wrap the underlying raw objects instead of directly work with them (which would lock in the use of the "plugin" to a certain technology, like Express).

Have you looked at my code samples. Maybe it makes it clearer what I want to suggest.

To come back to that, my initial intention was to set the http status depending on some outcome. It's not good enough for me to hardcode the status with a decorator.
To do that I had to create an interceptor that can't do what I want without going down to the raw objects. That's find as an option. But HTTP status, headers and content should be accessible without going down that hole (imho).

[jmcdo29]

Overall, I think this is something that would be incredibly useful. Unfortunately, to get to that point it may not be so simple. One of the big problems as you've noted is that Fastify uses a wrapper around the IncomingRequest object and the ServerResponse object. Nest provides ways to currently get those wrapper objects, but it does make making libraries more difficult (see Ogma and nestjs-throttler for two libraries that deal with these. It gets even more complicated when it comes to microservices, so we won't get into that discussion.

I'm trying to think if there could be a way that Nest creates it's own NestRequest object that (even further) wraps the req and res objects and provides an easy to use API like .header() or .status() to set headers and statuses. I'll keep thinking on if it's possible. Maybe a V8 goal to have for the framework. What are your thoughts @kamilmysliwiec @BrunnerLivio? I could see this being especially helpful when it comes to creating libraries, and if someone were to want to try switching underlying engines without _too_ much of a refactor.

[smolinari]

I think the only "proper" way to do it is stick to one http layer. Maybe, with its ecosystem doing so well, Nest's team could simply come up with their own layer (to also solve some of the issues you mention Jay), make it extensible, and look to the community to fill in the plug-in gaps of the other frameworks (oh, and make it as performant or better than Fastify 😁)?

Scott

[kamilmysliwiec]

We definitely want to avoid using our Response and Request wrappers and adding another abstraction layer on top of these objects, providing a new API that people would have to learn. If someone wants to dynamically change between different HTTP providers, I would suggest creating a lightweight wrapper on its own. We don't plan to provide something like this ootb

[mojo2012]

Yeah I have done that for my project. But it feels like this should be part of the framework.

[AKazm]

I agree on this being part of the framework itself.

I am currently diving in into NestJS using Fastify and Middlewares are giving me a headache, since neither FastifyRequest nor FastifyReply are available within a Middleware, which eventually results in rendering Middlewares useless for many use cases (since injected services rely on FastifyRequest and FastifyReply) they are made for.

Wrapping the framework's Request/Response types would help a lot here.