Nest: Support transaction per request with TypeORM

[shusson]

[x] Feature request

What is the motivation / use case for changing the behavior?

We would like to make all TypeORM database operations during a certain request/endpoint transactional. Based on https://github.com/typeorm/typeorm/issues/404#issuecomment-295855913 we could create a transactional entity manager and pass it around to all our services, but that doesn't feel very nestjs like. More recently @pleerock suggested the use of scoped services in https://github.com/typeorm/typeorm/issues/1895#issuecomment-380556498 which would enable a transaction-per-request and use the same transaction manager instance across all services without having to pass it around. A concrete example is in this doc.

Are scoped services something that the nestjs team has thought about implementing?

Is there a better way of handling TypeORM transactions in nestjs? Keeping in mind that custom repositories should not be singleton services and it is likely transaction decorators will be removed from TypeORM core.

For example:

UserController.ts

@Controller("/user")
export class UserController {
    constructor(
        private userService: UserService,
    ) {}

    @Post("/")
    @Transaction()
    async createNewUser(@Body() body: UserDTO) {        
        return await this.userService.createNewUser(body);
    }
}

UserService.ts

@Component()
export class UserService {
    constructor(private userRepository: UserRepository) {}

    async createNewUser(userDTO: UserDTO): Promise<User> {
        const user = new User();
        user.username = userDTO.username;
        ...

        const saved = await this.userRepository.save(user);

        await this.userRepository
            .createQueryBuilder()
            .relation(User, "departments")
            .of(saved)
            .add(userDTO.departmentIds);

        return saved;
    }
}

Where the dependency injected userRepository automagically is using the transactional entity manager under the hood.

[nicomouss]

As described in the example code provided in typeorm/typeorm#1895, you can use CLS (Continuous Local Storage) to store the transactional entity manager in the current CLS context, then retrieve that entity manager within your services and/or repositories, and work with it.

To do that, you can use the custom Transaction decorator provided in the example, and adapt it to your own use case. The custom Transaction decorator provided in the example is doing the following :
1) Opening a typeOrm transaction
2) Storing the transactional entity manager in the current CLS context
3) Calling your initial method (the one you decorated with the Transaction decorator)
4) Removing the transactional entity manager from the CLS context
5) Committing or Rolling Back the transaction

Using this approach you basically implement your own Transaction management system, encapsulating in this case the one provided by TypeOrm.

[shusson]

Thanks @nicomouss, @kamilmysliwiec is there any interest in implementing something like this in nestjs core?

[icepeng]

I think this feature should be implemented in https://github.com/nestjs/typeorm not the nestjs core.

[shusson]

@icepeng Yeah happy to move it, but I think the discussion has implications on nestjs core, i.e scoped services.

[kamilmysliwiec]

Hey @shusson,
In terms of scoped services, I have mixed feelings. Nest has a complex modules container, meaning, each time when the request hits our app, we would have to mount modules, resolve all relationships between providers (including async providers, such as database connection), and then attach them to the particular request. It's not only about resolving a set of services making it much more complicated.

However, what we definitely want to provide is a support for async hooks once they hit a stable release. With async hooks, we could create a kind of 'request bag' that could be easily accessed by each provider/controller/anything, in order to set/get any value that concerns only processed request. That will enable us to provide a way to handle with, for example, transactions-per-request feature. Nevertheless, the async-hooks are still in experimental phase.

[shusson]

Thanks for the update @kamilmysliwiec, async-hooks sounds like a promising way forward.

[kamilmysliwiec]

We'll get back to this feature once async-hooks functionality moves into stable API.

[odavid]

Hi All,
I've created a small project that adds a Spring Style Transactional Decorator and can control propagation.
See https://github.com/odavid/typeorm-transactional-cls-hooked

Hope this helps,
Ohad

[hidon84]

Hi All,
I've created a small project that adds a Spring Style Transactional Decorator and can control propagation.
See https://github.com/odavid/typeorm-transactional-cls-hooked

Hope this helps,
Ohad

I am finding this solution now!!! thanks alot!

[nicolae536]

Would this be a equvalent implementation ? I tested it and it works but I'm not sure if this works if multiple requests hit the server in the same time, I'm afraid to not make typeorm to do something like this.

-> reuqest -> change the entityManager1 -> start transaction -> await a select operation from the db
-> another request -> change the entityManager2 -> await another select operation from the db -> now the first select is ready to be continue so we resume that callback -> the current entityManger is not entityManager1 it's entityManager2

import { Injectable, NestInterceptor, ExecutionContext } from '@nestjs/common';
import { Observable, of } from 'rxjs';
import { EntityManager, Repository } from 'typeorm';

@Injectable()
export class DbTransactionInterceptor implements NestInterceptor {
    private nonTypesManager: any;

    constructor(private sharedEntityManager: EntityManager) {
        this.nonTypesManager = this.sharedEntityManager;
    }

    /**
     * IMPORTANT ANY INTERCEPTOR WHICH WANTS TO DO SOMETHING BEFORE THE REQUEST SHOULD
     * BE ADDED BEFORE THIS INTERCEPTOR
     * @param context
     * @param call$
     */
    intercept(context: ExecutionContext, call$: Observable<any>): Observable<any> | Promise<Observable<any>> {
        console.log('\r\n Interceptor 1');
        return this.sharedEntityManager.transaction(async (transactionManager) => {
            // In order to have a proper transaction synced with all repositories
            // we need to replace all entityManagers with the new manager created by the transaction
            const entityManagersMap = this.replaceEntityManagerWithActiveTransactionManager(transactionManager);

            // Wait for all context to execute
            const result = await call$.toPromise();

            // Request has ended so we need to put the managersBack
            this.replaceBackEntityMangers(entityManagersMap);
            console.log('\r\n After Interceptor 1\r\n');
            return of(result);
        });
    }

    private replaceEntityManagerWithActiveTransactionManager(transactionManager: EntityManager): Map<Repository<any>, any> {
        const entityManagersMap = new Map();

        this.nonTypesManager.repositories.forEach(repo => {
            entityManagersMap.set(repo, repo.manager);
            repo.manager = transactionManager;
        });

        return entityManagersMap;
    }

    private replaceBackEntityMangers(entityManagersMap: Map<Repository<any>, any>) {
        this.nonTypesManager.repositories.forEach(repo => {
            repo.manager = entityManagersMap.get(repo);
        });
    }
}

And then on the method

 @Post('...')
 @UseInterceptors(DbTransactionInterceptor)
    async editLater() {
             ...
    }

In order to not break how nest works the DbTransactionInterceptor should be declared as the last Interceptor

[nicolae536]

I tested the aproach above with Jmeter using concurent request and the application breaks, I also switched to cls-hooked. The described case it happens

_-> reuqest -> change the entityManager1 -> start transaction -> await a select operation from the db
-> another request -> change the entityManager2 -> await another select operation from the db -> now the first select is ready to be continue so we resume that callback -> the current entityManger is not entityManager1 it's entityManager2_

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.