Crud: Request must be send two times to make request filters applied

[maximelafarie]

According to my controller:

@Crud({
  model: {
    type: Lead,
  },
  query: {
    alwaysPaginate: true,
    join: {
      produit: {
        allow: ['nom'],
        eager: true
      }
    }
  },
  params: {
    id: {
      field: 'id',
      type: 'uuid',
      primary: true
    }
  }
})
@UseGuards(AuthGuard('jwt'))
@UseInterceptors(ClassSerializerInterceptor)
@Controller('leads')
@ApiUseTags('Leads')
@ApiBearerAuth()
export class LeadController implements CrudController<Lead> {
  constructor(public service: LeadService, public jwtUserService: JwtUserService) { }

  get base(): CrudController<Lead> {
    return this;
  }

  @Override()
  async getMany(
    @ParsedRequest() req: CrudRequest,
    @Headers() headers,
    @Query() query
  ) {
    const user = await this.jwtUserService.getUserFromJwt(headers.authorization);
    const hasQuery = (key: string) => !!query && !!query[key];

    if (user.role === 'user') {
      req.options.query.filter = {
        user: {
          $eq: user.id,
        }
      };
    }

    req.options.query.filter = {
      ...req.options.query.filter,
      ...hasQuery('produit') && { 'produit.id': { $in: query.produit } },
      ...hasQuery('statut') && { statut: { $in: query.statut } },
    };

    return this.base.getManyBase(req);
  }
}

Let's take the following routes (GET):
If I'm calling

http://localhost:3000/leads

It returns the normal collection of Lead resources:

{
  "data": [
    // Some data
  ],
  "count": 100,
  "total": 100,
  "page": 1,
  "pageCount": 1
}

But if I'm calling a first time:

http://localhost:3000/leads?statut%5B%5D=en_cours

It returns the same content as the global route /leads

But if I'm calling it a second time, it returns the expected result:

{
  "data": [],
  "count": 0,
  "total": 0,
  "page": null
}

I really don't understand why should I send the same request two times to get the right result.
Moreover, req.options.query.filter has always the right value each time I make a request. It's only the result that is incoherent. (Sorry if it's not related to the library, but I really don't get it).

[zMotivat0r]

So, you're saying that when you call this exact URL http://localhost:3000/leads?statut%5B%5D=en_cours two times - the result of the 1st call differs from the second?
If so, please show and compare the generated SQL of both calls.

[maximelafarie]

Thank you for quick reply @zMotivat0r!
Yes, that's exactly that! Yes sure, but do you have an easy way to log the generated SQL, because I only found tricky and awkward solutions to do that. 🤔Do I need to log a nestjsx/crud thing? Or more nestjs itself?

[maximelafarie]

Ok so the easiest way I found is to check in Postgresql directly the latests queries:

So the first call to this route generates this query:

SELECT COUNT(DISTINCT("Lead"."id")) as "cnt" FROM "lead" "Lead" LEFT JOIN "produit" "produit" ON "produit"."id"="Lead"."produit"

and the second (with the good result) generates:

SELECT COUNT(DISTINCT("Lead"."id")) as "cnt" FROM "lead" "Lead" LEFT JOIN "produit" "produit" ON "produit"."id"="Lead"."produit" WHERE ("Lead"."statut" IN ($1))

It's really strange since the two requests aren't the same, good catch @zMotivat0r but I don't know how and why it generates this.

[zMotivat0r]

Please confirm that your controller has query.statut in both calls. It seems that it doesn't. So just debug this.
Also, this construction seems to be not necessary:

req.options.query.filter = {
      ...req.options.query.filter,
      ...hasQuery('produit') && { 'produit.id': { $in: query.produit } },
      ...hasQuery('statut') && { statut: { $in: query.statut } },
    };

because you can do this by single query param that will be handled by the crud lib natively:

?s={"produit.id": {"$in": [...]}, "statut": {"$in": [...]}}

[maximelafarie]

Yes I can confirm: if I'm adding a console.log(query) I can see each time it has statut property.

Yes I know, that's awesome, thank you! It was just because the frontend is calling the route like that and I don't have any way to change it. 😕

[zMotivat0r]

So, even when you remove your construction and use ?s={"produit.id": {"$in": [...]}, "statut": {"$in": [...]}} instead, the generated SQL will still differ between the 1st and the 2nd call?

[maximelafarie]

No, I just tried it with my REST client, and if I'm calling the route the way you gave me above, all works perfectly right the first call!

[zMotivat0r]

It seems like a problem solved. But anyway, one more thing - you shouldn't change req.options if you want to change the incoming query string. You should change req.parsed instead.
Please close the issue if all works now as expected.

[maximelafarie]

Oh right I understand what you mean, so I can do the following:

1. take the query params sent by the frontend like I do
2. convert them like ?s={"given_param_1": {"$in": [...]}, "given_param_2": {"$in": [...]}}
3. pass this to the request as if it was quested this way

But how am I supposed to pass s={"given_param_1": {"$in": [...]}, "given_param_2": {"$in": [...]}} to the request the "typescript" way? 😉

Again, thank you for the help!

[zMotivat0r]

Nope. I meant you can totally remove this:

req.options.query.filter = {
      ...req.options.query.filter,
      ...hasQuery('produit') && { 'produit.id': { $in: query.produit } },
      ...hasQuery('statut') && { statut: { $in: query.statut } },
    };

and send this query from the frontend side s={"given_param_1": {"$in": [...]}, "given_param_2": {"$in": [...]}}

That's it.

[zMotivat0r]

Here are the scenario and explanations of your bug which is not a bug actually but a misunderstanding of crud lib internals which is totally fine. I'll explain more detailed:

1. Each request is handled by the CrudRequestInterceptor which then stores and passes a parsed request params/query in the req.parsed which is being used by CrudController methods.
2. You modify req.options.query.filter in the controller's method but the method will not handle it and will not take this filter into account because of all the logic related to the merging options.filter with the request query/params is already being done by the CrudRequestInterceptor. That's why a method cares only about req.parsed.search. And that's why during the 1st method call you're getting one result.
3. When you do this req.options.query.filter = ... you thus modify CrudOption object by a link. And during the 2nd method call your CrudOptions object is already modified with your changes and that's why the CrudRequestInterceptor parses a request with your query and creates a correct req.parsed.search object. That's why your 2nd method call will give you another result.

Hope this explanation helped

[maximelafarie]

Ok thank you for these explanations. 🖖