Tsoa: internal tsoa error: the metadata that was generated should have had nested property schemas since it’s for a nested object,however it did not

[simllll]

Sorting

• I'm submitting a ...

  • [X] bug report
  • [ ] feature request
  • [ ] support request

• I confirm that I

  • [X] used the search to make sure that a similar issue hasn't already been submit

Bug

I get the tsoa error mentioned in the titel. It tells me to open a bug report ;)

What I send to the API:

What the API has defined:

export type JobDescriptionInternalBlockType =
    | 'intro'
    | string;

export interface IAPIJobDescriptionBlock {
    language: string; // locale or language
    blocks: {
        header?: string;
        internalName?: JobDescriptionInternalBlockType;
        content: string;
    }[];
}

export interface TestInputDTO {
descriptionBlocks?: IAPIJobDescriptionBlock
}

The error happens here:

Steps to Reproduce

1. Create an interface with A PUT request like:

@Put('/test')
async test(
    @Body()
    body: TestInputDTO
) {
    // something
    return true;
}

1. Send request to the api
2. get error message

Context (Environment)

Version of the library: v2.5.3
Version of NodeJS: v12.6.0

• Confirm you were using yarn not npm: [ ] <-- i'm usign npm

[marius-usvat]

Am getting this error for union type arrays as well. The swagger seems to be generated correctly but the error gets thrown before reaching the controller.

interface Order {
  date: Date;
  delivery: Date;
  orderLines: Array<OrderLinePurchase | OrderLineRent | OrderLineService | OrderLineFee>;
}

I did not run into this error before since I used to handle this array via an intermediary object like the one below but I feel like it makes more sense to pass all the objects in the same array since they inherit a common type and I provided a discriminator type.

interface OrderLines {
  purchases: OrderLinePurchase[];
  rents: OrderLineRent[];
  services: OrderLineService[];
  fees: OrderLineFee[];
}

interface Order {
  orderLines: OrderLines;
}

I tried copying @WoH 's PR fix to my local repo but the error seems to persist. Am I approaching this the wrong way or does tsoa not support this yet?

[WoH]

My PR can't really help you here.

I am confused as to how your code may result is missing object literal metadata since I can't see any object literals in your code.

However, it seems like both issues are related in that they generate incorrect metadata for the array. If you could tell me about the metadata for your Model after applying my fix I'd be able to take a look.

(models[Order] in the routes.ts)

[marius-usvat]

Thanks for the edit I was a bit confused about where to find it. Here's the full error message for reference as well, it is indeed a different one, I got ahead of myself: internal tsoa error: the metadata that was generated should have had sub schemas since it’s for a union, however it did not.

"Order": {
    "properties": {
        "date": { "dataType": "datetime", "required": true },
        "delivery": { "dataType": "datetime", "required": true },
        "orderLines": { "dataType": "array", "array": { "dataType": "union" }, "required": true },
    },
    "additionalProperties": false,
}

[marius-usvat]

@WoH sorry, I didn't copy your PR properly, it does indeed generate the subschemas now and the error is gone

"Order": {
    "properties": {
        "date": { "dataType": "datetime", "required": true },
        "delivery": { "dataType": "datetime", "required": true },
        "orderLines": {
               "dataType": "array", 
               "array": {
                  "dataType": "union",
                  "subSchemas": [{ "ref": "OrderLinePurchase" }, { "ref": "OrderLineRent" }, { "ref": "OrderLineService" }, { "ref": "OrderLineFee" }]
               },
               "required": true
          },
     },
    "additionalProperties": false,
}

[WoH]

Given the error message I can see how it got mixed in there, don't worry (it's good it did since it's a different error caused by the same issue).
I had just assumed there was something else going wrong after applying my PR resulting in an error that I'd have to fix which is why I had to ask critical questions.

[dgreene1]

@marius-usvat & @simllll I released this feature/bug-fix in version v2.5.4. If you're enjoying using tsoa, please consider sharing it with friends/coworkers or on reddit, dev.to, or twitter. Thank you for being a part of our community! :)

[simllll]

Thanks a lot!
Yeah at least for Austria (Vienna) I can tell that I told already several people about it ;)
Great work so far, still some way to go, but I already love what we got! Really awesome project!

[dgreene1]

@simllll thank you. Yes it’s quit mature.

Thanks a lot! Yeah at least for Austria (Vienna) I can tell that I told already several people about it ;)
Great work so far, still some way to go, but I already love what we got! Really awesome project!

What features or aspects do you feel are lacking when you say there’s “still some way to go?”

[simllll]

What features or aspects do you feel are lacking when you say there’s “still some way to go?”

first things that come into my mind are:

• more focus on open api 3 (e.g. the defintions that are available right onw in the config, do not work for open api, you can easily overcome that by merging custom spec, but it's not so nice (
• middleware annotation feature (as soon as I find time I will work on this :))
• support of type aliases (e.g. we are working with a type called "TypeObjectId" which is a generic:
  export type TypeObjectId<T> = string & { type: T };
  right now this does not work at all, the generated swagger file is "not correct", and any client I generate out of it is therefore broken. (e.g. the validation fails because string is not an object... and the validator thinks typeobjectid is an object, which is actually wrong). (I know @WoH is already working on a type alias implementation, therefore I'm really curious for this and hopefuilly we can fix validation errors and other stuff after that too :))
• tuple support (would at least be super nice)
• some improved error handlings / error messages. RIght now I often get just a random error message where I can get no idea what is actually wrong and where it could be.
• some other small things in general with swagger / openapi (not directly related with tsoa). e.g. broken generated clients,...

But don't get me wrong, this project is amazing!! :)

[dgreene1]

Okay, yea I believe there are open issues and/or PRs for all of those. If you don’t see one, please open a new issue. Thank you for your feedback. :)

[WoH]

more focus on open api 3 (e.g. the defintions that are available right onw in the config, do not work for open api, you can easily overcome that by merging custom spec, but it's not so nice

Could you elaborate a bit more?

export type TypeObjectId<T> = string & { type: T };

My current implementation should handle that, if you want to make sure, I'd be very grateful if you could add tests on top of #485 to confirm your expectations.

some improved error handlings / error messages. RIght now I often get just a random error message where I can get no idea what is actually wrong and where it could be.

I have an error handler for ValidationErrors I should probably share with everyone, the error messages tsoa sets are pretty good.

[simllll]

more focus on open api 3

"servers" is the new way how you define your endpoints in open api. in swagger it was basehost and some others.
e.g. what we have in our tsoa file is something like this:

"spec": {
    "servers": [
        {
        "url": "html2pdf.network.hokify.com",
        "description": "hokify env"
        },
        {
        "url": "http://localhost:3007/html2pdf-service",
        "description": "local env"
        }
    ]
},

right now you only can achieve this by overrinding specs with this property, but no "native" way to specify an array of servers (or also just one server). (see https://github.com/lukeautry/tsoa/blob/f6c9820e3df10e15689e881f17aa274e03bd9f14/src/config.ts#L53 which does not generate servers for open api 3 if specified)
I also would vote for open api as default instead of swagger v2 ;)

export type TypeObjectId<T> = string & { type: T };
My current implementation should handle that, if you want to make sure, I'd be very grateful if you could add tests on top of #485 to confirm your expectations.

I will look into that. I will for sure test it :)

I have an error handler for ValidationErrors I should probably share with everyone, the error messages tsoa sets are pretty good.

Would be great, for example if I use a some unsupported "constellation" somehwere, you don't see where it is actually. (e.g. tuple...)

[WoH]

export type TypeObjectId = string & { type: T };

Is there anything you can actually assign here?

right now you only can achieve this by overrinding specs with this property, but no "native" way to specify an array of servers (or also just one server).

I see.

Would be great, for example if I use a some unsupported "constellation" somehwere, you don't see where it is actually. (e.g. tuple...)

I think I didn't understand your issue well before you said that. I assumed you were talking about the live validation failing, not the route / swagger generation.

I guess you could still easily throw the error with line information though.

[simllll]

Wow :) thanks already for another PR regarding error messages :+1: !

Regarding:
export type TypeObjectId<T> = string & { type: T };

during runtime this is just a string, but for explicit assingments we use a "wrapper" function that also validates the values, it looks something like this:

export const ObjectId = <T>(input: unknown): TypeObjectId<T> => {
    let myInput;
    // some validations, e.g.: 
    if (!isValidObjectId(myInput) && input !== 0) {
        throw new InvalidObjectId(`invalid object id: ${myInput}`);
    }

    return input as TypeObjectId<T>;
};

therefore in our code we use it either by getting this kind of object alredy e.g. from databse models where it is defined like this in the schema or by running it "through" the function ObjectId(...). My goal would be that I can define the TypeObjectId in my used tsoa models (actually I do this already for my regular models and have to copy them right now to change typeobjectid to string) and tsoa accepts it as simple "string". My second goal would be then afterwards to get some additional custom validtors running for this type, e.g. running the input through the ObjectId() method. (but this is another problem, I can live with that for now without it).

Does this answer help?

[WoH]

Does this answer help?

Somewhat. Your answer helps me ask a better question: Why is this not

export type TypeObjectId<T> = string | { type: T };

[simllll]

Well, for regular things this would probably be sufficient, but as we would like to force the ObjectId validation if we cast manually from string to objectid, it must be an "and" instead of an "or".

Let me show a simple example:

export type TypeObjectId<T> = string | { type: T }; // allows accidental string assignment
// export type TypeObjectId<T> = string & { type: T }; // let the string assingment fail

export const ObjectId = <T>(input: unknown): TypeObjectId<T> => {
    return input as TypeObjectId<T>;
}

interface House {
    _id: TypeObjectId<House>;
    color: string;
}

interface Garden {
    _id: TypeObjectId<Garden>;
    squaremeters: number;
}

const house = ObjectId<House>('12345');
const garden = ObjectId<Garden>('12345');

function getHouse(houseId: TypeObjectId<House>) {
    console.log("get house with id: " + houseId);
}

function getGarden(gardenId: TypeObjectId<Garden>) {
    console.log("get garden with id: " + gardenId);
}

getHouse(house); // should work
getHouse(garden); // should fail

getGarden(house); // should work
getGarden(garden); // should fail

getHouse('12123123123a'); // should fail
getGarden('asfasdfasdfsdfas'); // should fail

But I understand that this could be a problem in TSOA, at least as long as we have no way to "insert" custom transformations / validtions for input variables. Which is probably even a very special edge case...

[WoH]

But I understand that this could be a problem in TSOA, at least as long as we have no way to "insert" custom transformations/validations for input variables. Which is probably even a very special edge case...

Fun fact, it isn't when it comes to swagger generation. I just think nothing you could send to the server should ever get past my ValidationService checks.

[dgreene1]

@simllll the idea for custom transforms sounds fun. Would you mind moving that to a new issue?

[simllll]

Fun fact, it isn't when it comes to swagger generation. I just think nothing you could send to the server should ever get past my ValidationService checks.

haha, interesting. but unsure if I quite understand it correclty what that means. The swagger defintion can be generated but when someone wants to send data to the api it gets rejected? Or does it just "slip" through as string and the validation is just against a valid string (which is probably no validation at all?). If it's the latter case, then this would be totally fine for me.
If it's the first one, then I would need a "custom tranformer" I was mentionend before.

I can offer to look into that as soon as the type alias support is there :) (e.g. like strings are converted to numbers right now, I would like the ability to add a custom tranformation for a specifiy key, but this transformation can also fail and will be handled like the current validation erros)

[WoH]

can be generated but when someone wants to send data to the api it gets rejected?

Yes, since tsoa errors when you send data that does not match what you asked for.
Basically, the ValidationService is a Swagger validation layer that checks if the data matches the spec, which will never be possible.

[simllll]

Basically, the ValidationService is a Swagger validation layer that checks if the data matches the spec, which will never be possible.

but wouldn't #496 solve this?

[WoH]

I think the way to do this is use a export type TypeObjectId<T> = string | { type: T }; and reassign to a export type TypeObjectId<T> = { type: T }; or whatever you want to cast this to in your code. I personally don't think it's up to tsoa to handle that.

[simllll]

I fully agree that this case should not be handled by tsoa out of the box, but in case you will allow a custom transformer / validator to be applied, anyone could handle this kind of things exactly at the right place (e.g. to the type cast inside the custom transformer).