Tsoa: Some issues with typegoose

[Eywek]

Sorting

• I'm submitting a ...

  • [ ] bug report
  • [ ] feature request
  • [x] support request

• I confirm that I

  • [x] used the search to make sure that a similar issue hasn't already been submit

Context (Environment)

Version of the library: v3.0.3
Version of NodeJS: v12.14.1

• Confirm you were using yarn not npm: [x]

Ask

Hello,

I'm trying to use [email protected] and I'm having some issues when I'm trying to generate the swagger schema.

I have a typegoose model like that:

export class Datasource {

  @prop({
    required: [true, 'missing companyId']
  })
  public companyId!: ObjectId

  @prop({
    trim: true,
    required: [true, 'NameEmpty'],
    default: () => `Data (${moment().format('L')})`
  })
  public name!: string

  @prop({
    required: true,
    default: 0
  })
  public contentVersion!: number

  @arrayProp({
    default: [],
    itemsRef: DatasourceVersion
  })
  public versions!: Ref<DocumentType<DatasourceVersion>>[]

  ...

and I'm encountering the error:

Generate swagger error.
 SyntaxError: Unexpected token ` in JSON at position 0
    at JSON.parse (<anonymous>)
    at TypeResolver.getNodeExample (/Users/valentin/Work/datasources-v2/packages/frontend/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:813:25)

because the DocumentType type is exported like this:

/**
 * Get the Type of an instance of a Document with Class properties
 * @public
 * @example
 * ```ts
 * class Name {}
 * const NameModel = Name.getModelForClass(Name);
 *
 * const t: DocumentType<Name> = await NameModel.create({} as Partitial<Name>);
 * ```
 */
export declare type DocumentType<T> = T extends Base ? Omit<mongoose.Document, '_id'> & T : mongoose.Document & T;

I'm wondering if there is anyway to avoid that problem? In my case I don't care about JSDoc comments, maybe there is a way to disable the detection?

EDIT:

• Another issue, I can't use Omit<> (in my controller response) because it seems to be defined in @types/mongodb causing this error:

Error: Multiple matching models found for referenced type Omit; please make model names unique. Conflicts found: ".../node_modules/@types/mongodb/index.d.ts"; ".../node_modules/@types/mongodb/index.d.ts".

• And when I generate a swagger schema with my Datasource class, ObjectId is defined like that:

 ObjectId:
            description: 'A class representation of the BSON ObjectId type.'
            properties:
                generationTime:
                    type: number
                    format: double
                    description: 'The generation time of this ObjectId instance'
                cacheHexString:
                    type: boolean
                    description: 'If true cache the hex string representation of ObjectId'
            required:
                - generationTime
            type: object
            additionalProperties: true

instead of a string, but I can't use string type in my typegoose model because it will break my types in my backend. Any solutions for this?

Thank you

[WoH]

We should ignore @example if you can't transform it to a JSON object, that should address the issue. v3.0.4 fixes that.

Regarding Omit: The most pragmatic approach is to redefine:

/**
 * @tsoaModel
 */
type Omit<T, K> = Pick<T, Exclude<keyof T, K>>;

So we can make sure that one is used.
In the future, I'm looking at an approach to Entity resolution that uses a mechanism that's more like the one TS exposes in vscode as "Jump to declaration", which may solve these issues, but for now I'll have to suggest the workaround above.

Object id is declared here, and the spec created from that class seems to be the one you're getting. What's the issue with that? I'm not too familiar with MongoDB, maybe you can help me out. What happens when you serialize?

[Eywek]

Thank you for the fix, it worked

I've tried to redefine Omit as type OmitVersion<T, K> = Pick<T, Exclude<keyof T, K>> and this error is triggered:

Generate swagger error.
 Error: Unknown type: LiteralType.
At: /Users/valentin/Work/datasources-v2/packages/frontend/src/controllers/datasources/analyzer.ts:21:21.
This was caused by 'Promise<OmitVersion<Datasource, 'versions'>>'

When and ObjectId is json stringified it becames a string, I've tried to extends ObjectId like that but it doesn't work:

interface Test extends ObjectId {
  toJSON (): string,
  toString (): string
}

[WoH]

First of all, thanks for testing the v3 prereleases and reporting issues. I'll try to make sure we get all the issues solved before this hits the stable channel.

Regarding your Test type: Extending this way will not work because tsoa can't know this method is called before returning.
The dirty hack you could try would be defining a

/**
 * A serialized ObjectId
 * @tsoaModel
 */
type ObjectId: string;

The better approach however would be to declare an interface something like

export interface SerializedDatasource extends OmitVersion<DataSource, 'companyId'> {
  companyId: string
}

Maybe @simllll has some insight/tips? I believe he faced a similar issue when upgrading tsoa.

I'll try to reproduce error with the LiteralType, thanks for reporting that.

[WoH]

@Route()
export class GetController {
  @Get('Omissions')
  public async getOmittedModel(): Promise<OmitAlias<PrivateModel, 'stringPropDec1'>> {
    return {} as any;
  }
}

type OmitAlias<T, K> = Pick<T, Exclude<keyof T, K>>;

seems to not cause any issues. Would you mind sharing your entire class?

We should be able to resolve LiteralTypes, so I assume we are not expecting it to occur in one of the properties in a way we are not prepared for. If you're using any types from mongo, I'll be able to track them down from there.

[Eywek]

For the SerializedDatasource the output is:

SerializedDatasource:
            properties:
                companyId:
                    type: string
            required:
                - companyId
            type: object
            additionalProperties: true

where other properties are missing

---

My controller:

import { Types, of, httpErrors, Datasource, DatasourceVersion } from '@rlvt/lib'

type OmitK<T, K> = Pick<T, Exclude<keyof T, K>>
// export interface SerializedDatasource extends OmitK<Datasource, 'companyId'> {
//   companyId: string
// }


/**
 * Analyze source and update datasource with sample, csvDelimiter and fielfsMap
 */
@Route()
export class AnalyzerController extends Controller {

  @Post('{id}/analyzer')
  // @Response<{ data: Types.RawDatasource & Types.RawDatasourceVersion }>('datasource')
  public async analyze (
    @Request() req: express.Request,
    @Path('id') id: string,
    @Header('x-companyid') companyId: string
  ): Promise<OmitK<Datasource, 'versions'>> {
    const datasource = await req.models.Datasource.findOne({
      _id: mongoose.Types.ObjectId(id),
      companyId
    }).populate({
      path: 'versions',
      match: { state: { $in: ['DRAFT', 'LIVE'] } }
    }).exec()

   ...
  }
}

@rlvt/lib index (we're using lerna with typescript paths):

export * from './models/datasources/index'
export * from './models/datasource_versions/index'

models/datasources/index:

import { prop, getModelForClass, modelOptions, index, arrayProp, DocumentType, Ref, pre } from '@typegoose/typegoose'
import { hidden } from '../datasource_versions/utils'
import { DatasourceVersion, DatasourceVersionModel } from '../datasource_versions'
import { ObjectId } from 'mongodb'

@index({ companyId: 1 })
@modelOptions({
  schemaOptions: {
    toJSON: {
      virtuals: true,
      versionKey: false,
      transform: hidden.bind(this, ['_id', 'versions'])
    },
    timestamps: true
  }
})
@pre<Datasource>('remove', function (next) {
  return DatasourceVersionModel.deleteMany({
    _id: this.versions
  }).exec(next)
})
export class Datasource {

  @prop({
    required: [true, 'missing companyId']
  })
  public companyId!: ObjectId

  @prop({
    trim: true,
    required: [true, 'NameEmpty'],
    default: () => `Data (${moment().format('L')})`
  })
  public name!: string

  @prop({
    required: true,
    default: 0
  })
  public contentVersion!: number

  @arrayProp({
    default: [],
    itemsRef: DatasourceVersion
  })
  public versions!: Ref<DocumentType<DatasourceVersion>>[]
}

(I've stripped out some of our logic but I think this will be enough)

The typeNode variable from tsoa/dist/metadataGeneration/typeResolver.js:524:19:

NodeObject {
  pos: 856,
  end: 867,
  flags: 0,
  modifierFlagsCache: 0,
  transformFlags: 536870913,
  parent: NodeObject {
    pos: 839,
    end: 868,
    flags: 0,
    modifierFlagsCache: 0,
    transformFlags: 536870913,
    parent: NodeObject {
      pos: 830,
      end: 869,
      flags: 0,
      modifierFlagsCache: 0,
      transformFlags: 536870913,
      parent: [NodeObject],
      kind: 169,
      typeName: [IdentifierObject],
      typeArguments: [Array]
    },
    kind: 169,
    typeName: IdentifierObject {
      pos: 839,
      end: 844,
      flags: 0,
      modifierFlagsCache: 0,
      transformFlags: 536870912,
      parent: [Circular],
      kind: 75,
      escapedText: 'OmitK',
      flowNode: [Object]
    },
    typeArguments: [
      [NodeObject],
      [Circular],
      pos: 845,
      end: 867,
      transformFlags: 536870913
    ]
  },
  kind: 187,
  literal: TokenObject {
    pos: 856,
    end: 867,
    flags: 0,
    modifierFlagsCache: 0,
    transformFlags: 536870912,
    parent: [Circular],
    kind: 10,
    text: 'versions'
  }
}

[WoH]

I forgot to confirm: You're not using custom templates, right?

[Eywek]

Nope, my config:

{
    "compilerOptions": {
        "baseUrl": "<removed>/packages/frontend/src",
        "paths": {
          "@rlvt/lib": [ "<removed>/packages/lib/src" ]
        }
      },
      "swagger": {
        "basePath": "/",
        "entryFile": "<removed>/packages/frontend/src/index.ts",
        "specVersion": 3,
        "outputDirectory": "<removed>/packages/frontend/src/routes",
        "controllerPathGlobs": [
          "<removed>/packages/frontend/src/controllers/datasources/*.ts"
        ],
        "version": "unknown",
        "yaml": true
      },
      "routes": {
        "basePath": "/",
        "entryFile": "<removed>/packages/frontend/src/index.ts",
        "routesDir": "<removed>/packages/frontend/src/routes"
      }
}

[Eywek]

Hey,
Do you have any news on this?
Thank you

[WoH]

Yeah. I was unable to reproduce the issue given the information I have so far. Can you remove the routes.ts and see if that changes anything?

Also, set the controllerPathGlob for both routes and swagger please.

[Eywek]

I still have the issue when I'm only trying with:

interface A { stringPropDec1: B }
interface B { foo: string }
@Route()
export class GetController {
  @Post('Omissions')
  public async getOmittedModel(): Promise<OmitAlias<A, 'stringPropDec1'>> {
    return {} as any;
  }
}

type OmitAlias<T, K> = Pick<T, Exclude<keyof T, K>>;

➜  frontend git:(wip-swagger) ✗ ./node_modules/.bin/tsoa swagger
Generate swagger error.
 Error: Unknown type: LiteralType.
At: /Users/valentin/Work/datasources-v2/packages/frontend/src/controllers/datasources/analyzer.ts:17:17.
This was caused by 'Promise<OmitAlias<A, 'stringPropDec1'>>'
    at new GenerateMetadataError (/Users/valentin/Work/datasources-v2/packages/frontend/node_modules/tsoa/dist/metadataGeneration/exceptions.js:21:28)
    at TypeResolver.getAnyTypeName (/Users/valentin/Work/datasources-v2/packages/frontend/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:523:19)
    at /Users/valentin/Work/datasources-v2/packages/frontend/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:502:45
    at Array.reduce (<anonymous>)
    at TypeResolver.getTypeName (/Users/valentin/Work/datasources-v2/packages/frontend/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:495:14)
    at TypeResolver.getReferenceTypeOrEnumType (/Users/valentin/Work/datasources-v2/packages/frontend/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:429:40)
    at TypeResolver.resolve (/Users/valentin/Work/datasources-v2/packages/frontend/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:228:40)
    at TypeResolver.resolve (/Users/valentin/Work/datasources-v2/packages/frontend/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:207:136)
    at MethodGenerator.Generate (/Users/valentin/Work/datasources-v2/packages/frontend/node_modules/tsoa/dist/metadataGeneration/methodGenerator.js:53:76)
    at /Users/valentin/Work/datasources-v2/packages/frontend/node_modules/tsoa/dist/metadataGeneration/controllerGenerator.js:41:58

On which typescript version are you trying? I'm on 3.7.2.

---

Have you any idea for the SerializedDatasource issue?

Thank you

[WoH]

Typescript version does not matter. We internally use 3.8 to parse the code but that won't matter.
The SerializedDatasource issue is a result of the Omit issue I'm prettu sure.
If my suggestions from https://github.com/lukeautry/tsoa/issues/618#issuecomment-603188505 won't work you're probably out of luck unless someone can provide a minimal reproduction repo I can debug.

Especially considering the code I ran through the tests causes an issue in your code.

[Eywek]

You can see the bug with this repo: https://github.com/Eywek/tsoa-bug
Just clone it, run yarn and yarn bug, you should see the error.

[WoH]

Cloned the repo and I can confirm it happens. However, I was under the assumption you were using a beta version (some 3.0.x). Looking at the pck.json, tsoa is @ "^2.5".

So just to confirm this doesn't happen in v3 I ran:

tsoa-bug on  master via ⬢ v12.16.1 took 7s 
➜ yarn bug
yarn run v1.21.1
warning package.json: No license field
$ tsoa swagger
Generate swagger error.
 Error: Unknown type: LiteralType.
At: src/controllers/test.ts:8:8.
This was caused by 'Promise<OmitAlias<A, 'stringPropDec1'>>'
    at new GenerateMetadataError (/home/woh/Code/Playground/tsoa-bug/node_modules/tsoa/dist/metadataGeneration/exceptions.js:21:28)
    at TypeResolver.getAnyTypeName (/home/woh/Code/Playground/tsoa-bug/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:523:19)
    at /home/woh/Code/Playground/tsoa-bug/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:502:45
    at Array.reduce (<anonymous>)
    at TypeResolver.getTypeName (/home/woh/Code/Playground/tsoa-bug/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:495:14)
    at TypeResolver.getReferenceTypeOrEnumType (/home/woh/Code/Playground/tsoa-bug/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:429:40)
    at TypeResolver.resolve (/home/woh/Code/Playground/tsoa-bug/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:228:40)
    at TypeResolver.resolve (/home/woh/Code/Playground/tsoa-bug/node_modules/tsoa/dist/metadataGeneration/typeResolver.js:207:136)
    at MethodGenerator.Generate (/home/woh/Code/Playground/tsoa-bug/node_modules/tsoa/dist/metadataGeneration/methodGenerator.js:53:76)
    at /home/woh/Code/Playground/tsoa-bug/node_modules/tsoa/dist/metadataGeneration/controllerGenerator.js:41:58
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

tsoa-bug on  master via ⬢ v12.16.1 took 2s 
➜ yarn run tsoa --version
yarn run v1.21.1
warning package.json: No license field
$ /home/woh/Code/Playground/tsoa-bug/node_modules/.bin/tsoa --version
2.5.13
Done in 0.74s.

tsoa-bug on  master via ⬢ v12.16.1 
➜ yarn add tsoa@v3beta
yarn add v1.21.1
warning package.json: No license field
warning No license field
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
warning No license field
success Saved 2 new dependencies.
info Direct dependencies
└─ [email protected]
info All dependencies
├─ [email protected]
└─ [email protected]
Done in 6.38s.

tsoa-bug on  master [!] via ⬢ v12.16.1 
➜ yarn run tsoa --version
yarn run v1.21.1
warning package.json: No license field
$ /home/woh/Code/Playground/tsoa-bug/node_modules/.bin/tsoa --version
3.0.4
Done in 0.97s.

tsoa-bug on  master [!] via ⬢ v12.16.1 
➜ yarn run tsoa swagger  
yarn run v1.21.1
warning package.json: No license field
$ /home/woh/Code/Playground/tsoa-bug/node_modules/.bin/tsoa swagger
Done in 2.71s.

[Eywek]

That's right sorry, I've probably got lost with all my tests and I've probably rollback to version 2 without noticing.
I have a new bug with methods on my model, you can reproduce with https://github.com/Eywek/tsoa-bug (Note: you'll need to update absolute paths in packages/api/tsoa.json)

Thank you

[WoH]

Your config seems off, changing the compilerOptions tsoa is supposed to use to:

  "compilerOptions": {
    "rootDirs": ["src", "package.json"],
    "outDir": "./build",
    "baseUrl": "../../packages/",
    "paths": {
      "@rlvt/lib": ["lib/src"],
      "@rlvt/api": ["api/src"]
    }
  },

resolves the issue.
PSA: Serializing a class with public methods will not work, either ignore that or remove the method.

[Eywek]

Right, I've fixed the config (https://github.com/Eywek/tsoa-bug/commit/1e1a3efc546bf7bd4d6069c0bdcedaed9f0b0cc9), but the bug is still there. I only want to ignore my methods in my swagger

[WoH]

I took the time to debug the repo twice, pointed out a possible issue and ran it through spec generation, which confirmed the generation was fine. I think this support request should therefore considered solved.
If there is another concrete issue, feel free to open another support request, but this one has gone way beyond the request it was opened for making it hard to follow.

[Eywek]

Yes, I'm sorry, I've done mistakes and took your time for a stupid issue about my tsoa version.
But right now I'm just trying to use your library and find out potential bugs that you may want to fix. I've tried to give you a repo where my issues are reproducible. And I've taken time on my side to debug as much as I can.

Last question, I've not understood if my method error is on purpose and it's not supported (and I need to use Omit to remove them) or it's happening because of my setup?

Thank you

[WoH]

If you look at the output, tsoa is now telling you there's an issue because we can't document the class. I told why it's happening (we can't document a method and probably shouldn't since it won't be serialized) and how to fix it ("either ignore that or remove the method").
I'm saying that because I already ignored the method (using @ignore) to verify the generation was fine. Therefore it's very frustrating for me if you're telling me "the bug is still there" when I once again confirmed it's not. It's very hard to answer your questions because they're based on the info you're providing, which is very unprecise (version, error output), is which is why I decided it's not useful to continue on with an already very convoluted conversation.

[Eywek]

Okayy, I understand now, thank you. I've misunderstood first because as you replied with a config, I was thinking that was the solution to my issue and that your PSA was only a warning to tell me that my method will not be documented (which wasn't an issue in my case). So when I've tried with the new config (and without the @ignore which I've not thinking of), the error was still there and I wasn't understanding how to resolve it.

About unprecise informations, that's why I've provided you a repo to be able to reproduce easily.
Anyway, that was a misunderstanding. Thank you for your time.