Tsed: [BUG] @CollectionOf Map field not serialized, @Ignore not working.

[vorph1]

Information

• Version: 6.14.2
• Packages:

  "dependencies": {
    "@tsed/ajv": "6.14.2",
    "@tsed/common": "6.14.2",
    "@tsed/core": "6.14.2",
    "@tsed/di": "6.14.2",
    "@tsed/exceptions": "6.14.2",
    "@tsed/json-mapper": "6.14.2",
    "@tsed/mongoose": "6.14.2",
    "@tsed/passport": "6.14.2",
    "@tsed/platform-express": "6.14.2",
    "@tsed/schema": "6.14.2",
    "@tsed/swagger": "6.14.2",
    "@tsed/testing-mongoose": "6.14.2",
    "ajv": "6.12.6",
    "argon2": "^0.27.0",
    "body-parser": "1.19.0",
    "compression": "1.7.4",
    "cookie-parser": "1.4.5",
    "cors": "2.8.5",
    "cross-env": "7.0.3",
    "dotenv-flow": "^3.2.0",
    "express": "4.17.1",
    "helmet": "4.2.0",
    "jsonwebtoken": "^8.5.1",
    "lodash": "^4.17.20",
    "method-override": "3.0.0",
    "mongoose": "5.11.5",
    "passport": "0.4.1",
    "passport-custom": "^1.1.1",
    "passport-jwt": "^4.0.0"
  },
  "devDependencies": {
    "@tsed/cli-plugin-eslint": "2.7.5",
    "@tsed/cli-plugin-jest": "2.7.5",
    "@tsed/cli-plugin-mongoose": "2.7.5",
    "@tsed/cli-plugin-passport": "2.7.5",
    "@tsed/testing-mongoose": "6.14.2",
    "@types/compression": "^1.7.0",
    "@types/cookie-parser": "^1.4.2",
    "@types/cors": "^2.8.8",
    "@types/dotenv-flow": "^3.1.0",
    "@types/express": "4.17.9",
    "@types/helmet": "4.0.0",
    "@types/jest": "26.0.17",
    "@types/jsonwebtoken": "^8.5.0",
    "@types/lodash": "^4.14.165",
    "@types/method-override": "0.0.31",
    "@types/multer": "^1.4.4",
    "@types/node": "14.14.10",
    "@types/passport": "1.0.4",
    "@types/passport-jwt": "^3.0.3",
    "@types/supertest": "2.0.10",
    "@typescript-eslint/eslint-plugin": "4.9.1",
    "@typescript-eslint/parser": "4.9.1",
    "concurrently": "5.3.0",
    "eslint": "7.15.0",
    "eslint-config-prettier": "7.0.0",
    "eslint-plugin-prettier": "3.2.0",
    "husky": "^4.3.0",
    "jest": "26.6.3",
    "lint-staged": ">=10",
    "nodemon": "2.0.6",
    "prettier": "^2.2.0",
    "supertest": "6.0.1",
    "ts-jest": "26.4.4",
    "ts-node": "9.1.1",
    "typescript": "4.1.2"
  },

Hi,
I'm having some issues with the model below

• @CollectionOf(UserModuleData) decorated field is not being serialized (empty {} returned)
• @Ignore((value, ctx) => ctx.endpoint) doesn't seem to have an effect on a Map field

I have verified that the data is stored and retrieved correctly, only serialization to json seems to be problematic.

Example

import {
  CollectionOf,
  Default,
  Email,
  Format,
  Ignore,
  MinLength,
  Pattern,
  Required,
} from '@tsed/schema';
import {
  Model,
  MongooseIndex,
  ObjectID,
  Ref,
  Schema,
  Unique,
} from '@tsed/mongoose';

import { Document } from 'mongoose';
import { Module } from '../modules';
import rx from '../utils/rx';

@Schema({
  schemaOptions: { _id: false },
})
export class UserModuleData {
  @(CollectionOf(Number).MinItems(1))
  roles: number[];
}

@Model()
@MongooseIndex({ 'dataScope.$**': 1 }) // wildcard
export class UserModel {
  static PASSWORD_PATTERN = rx()`
    ^                           # Start anchor
      (?=.*[A-Z].*[A-Z])          # Ensure string has two uppercase letters.
      (?=.*[!@#$&*])              # Ensure string has one special case letter.
      (?=.*[0-9].*[0-9])          # Ensure string has two digits.
      (?=.*[a-z].*[a-z].*[a-z])   # Ensure string has three lowercase letters.
      .{8,}                       # Ensure string is of length 8+.
    $                           # End anchor.
  `;

  @ObjectID('id')
  _id: string;

  @Required()
  @MinLength(3)
  name: string;

  @Unique()
  @Email()
  @Required()
  email: string;

  @Pattern(UserModel.PASSWORD_PATTERN)
  @Required()
  @Ignore((value, ctx) => ctx.endpoint)
  password: string;

  @CollectionOf(UserModuleData)
  data: Map<string, UserModuleData>;

  @Ref(UserModel)
  @Ignore((value, ctx) => ctx.endpoint)
  dataScope: Map<string, Ref<UserModel>>;

  @Format('date-time')
  @Default(Date.now)
  createdAt: Date = new Date();
}

export type UserModelDoc = UserModel & Document;

Usage in service

  async upsertUser(user: Partial<UserModel>) {
    const u = user._id ? await this.model.findById(user._id) : new this.model();

    if (!user._id && !user.password)
      throw new Error('Missing password at creation');

    if (!u) throw new Error('User not found');

    u.set({ ...user });

    if (user.password) u.set({ password: await Argon2.hash(user.password) });

    await u.save();

    return u;
  }

Endpoint call

Acceptance criteria

• [x] @CollectionOf(Schema) Map<string,Schema> field is serialized correctly
• [x] @Ignore(...) Map<string,Ref<Model>> field is ignored correctly

[Romakita]

Hello @vorph1

This issue was complicated to fix, sorry :). The problem come from mongoose itself, because, mongoose doesn't return a real instance of the class. So the nested model/schema aren't correctly mapped.

I added the following Caveat to explain the problem:

Mongoose doesn't return a real instance of your class. If you inspected the returned item by one of mongoose's methods,
you'll see that the instance is as Model type instead of the expected class:

import {Inject, Injectable} from "@tsed/common";
import {MongooseModel} from "@tsed/mongoose";
import {Product} from "./models/Product";

@Injectable()
export class MyRepository {
  @Inject(Product) 
  private model: MongooseModel<Product>;

  async find(query: any) {
    const list = await this.model.find(query).exec();

    console.log(list[0]); // Model { Product }

    return list;
  }
}

There is no proper solution currently to have the expected instance without transforming the current instance to
the class with the @@deserialize@@ function.

To simplify this, Ts.ED adds a toClass method to the MongooseModel to find, if necessary, an instance of type Product.

import {Inject, Injectable} from "@tsed/common";
import {MongooseModel} from "@tsed/mongoose";
import {Product} from "./models/Product";

@Injectable()
export class MyRepository {
  @Inject(Product) 
  private model: MongooseModel<Product>;

  async find(query: any) {
    const list = await this.model.find(query).exec();

    console.log(list[0]); // Model { Product }
    console.log(list[0].toClass()); // Product {}

    return list;
  }
}

Ts.ED will serialize correctly the mongoose model instance now but for developer it's important to understand the fact that mongoose return a MongooseModel instance and some properties haven't the expected type (like data: Map<string, UserModuleData>) contrary to what is indicated by typescript.

I added a toClass method to retrieve the expected types, if you have to manipulate the data property in your code.

If you have any question tell me :)
See you

[Romakita]

:tada: This issue has been resolved in version 6.14.4 :tada:

The release is available on:

• v6.14.4
• GitHub release

Your semantic-release bot :package::rocket:

[vorph1]

Hi,
Thank you for your help! The first issue is resolved the second however is not - sorry, I probably should have reported those two as separate tickets.

If I just log ctx.endpoint like shown below it is undefined for the @Ref field, not sure if that's specific to @Ref decorator or Map type.

  @Pattern(UserModel.PASSWORD_PATTERN)
  @Required()
  @Ignore((value, ctx) => {
    console.log('Password', ctx.endpoint);
    return ctx.endpoint;
  })
  password: string;

  @CollectionOf(UserModuleData)
  data: Map<string, UserModuleData>;

  @Ignore((value, ctx) => {
    console.log('Ref', ctx.endpoint);
    return ctx.endpoint;
  })
  @Ref(UserModel)
  dataScope: Map<string, Ref<UserModel>>;

[Romakita]

  @Ignore((value, ctx) => {
    console.log('Ref', ctx.endpoint);
    return ctx.endpoint;
  })
  @Ref(() => UserModel) // to be fixed
  @CollectionOf(() => UserModel)
  dataScope: Map<string, Ref<UserModel>>;

[Romakita]

@vorph1 The next release will fix your second issue :)
Sorry for the delay!

[Romakita]

:tada: This issue has been resolved in version 6.17.2 :tada:

The release is available on:

• v6.17.2
• GitHub release

Your semantic-release bot :package::rocket: