Tsed: [V6][BUG/QUESTION] request.file is undefined in >= 6.20.0

[silveoj]

These 2 examples don't work starting with v 6.20.0 https://github.com/TypedProject/tsed/compare/v6.19.2...v6.20.0

request.file is undefined

Example 1

@Post('/my-image/:id')
@Use(upload.single('myimage')) // frontend sends me `myimage` field
public uploadMyIng(
  @Req() request,
) {
  // request.file is undefined
}

Example 2

private getFileConfig(myType: myEnum) {
  return {
    storage: multer.diskStorage({
      destination: (req: Express.Request, _fileItem, cb) => {
        const path = 'resolve path here depends on myType param';
        cb(null, path);
      },
      filename: (req, fileItem, callback) => {
        // handle filename
      },
    }),
    fileFilter: (req, fileItem, callback) => {
      // filter file
    },
  };
}

  @Post('/upload')
  @MulterOptions(getFileConfig(myEnum.A)) // now pass config here
  public async uploadDoc(
    @MultipartFile('file') file: PlatformMulterFile, // frontend sends me `file` field
    @BodyParams('id') id: number,
  ) {
    // request.file is undefined
  }

Thanks.

[Romakita]

Hello @silveoj
Have you tried with the latest version? Because I’ve recently fixed the issue about multer.

see you

[silveoj]

Hello @Romakita I use 6.22.2

    "@tsed/ajv": "6.22.2",
    "@tsed/common": "6.22.2",
    "@tsed/core": "6.22.2",
    "@tsed/di": "6.22.2",
    "@tsed/exceptions": "6.22.2",
    "@tsed/platform-express": "6.22.2",
    "@tsed/schema": "6.22.2",
    "@tsed/swagger": "6.22.2",
    "@tsed/typeorm": "6.22.2",

    "express": "4.17.1",
    "express-http-context": "1.2.4",
    "express-session": "1.17.1",

released 6 days ago
request.file is undefined

[silveoj]

Maybe do I something missed? Maybe AJV is wrong?

I'm testing v. 6.19.2. Multer memoryStorage and diskStorage work.

1. v. 6.19.2 (because 6.22.2 doesn't work for me)

@MulterOptions(getFileConfig(myEnum.A))
@MyFileUploadDecorator()
public uploadMyFile(
  @BodyParams('type') type: keyof typeof MyEnum, // now validation fails Cannot read property 'map' of null
) {
  //
}

keyof typeof MyEnum brokes validation. I can use quick fix @BodyParams('type') type: string. But it's not typescript way)

'TypeError: Cannot read property 'map' of null
    at AjvService.mapErrors (\node_modules\@tsed\ajv\src\services\AjvService.ts:61:8)
    at AjvService.validate (\node_modules\@tsed\ajv\src\services\AjvService.ts:45:20)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at ValidationPipe.transform (\node_modules\@tsed\common\src\mvc\pipes\ValidationPipe.ts:71:5)
    at handleError (\node_modules\@tsed\common\src\platform\services\PlatformHandler.ts:350:16)

1. Also I have decorator to check if file exist

/**
 * Check min size and other properties which Multer can't
 */
@Middleware()
export class MyFileUploadMiddleware implements IMiddleware {
  public use(
    @Request() request: Express.Request,
    @Next() next: Express.NextFunction,
    @MultipartFile('file') files: PlatformMulterFile[],
  ) {
    if (!files || !files.length) {
      throw new BadRequest("Can't find file field...");
    }

    // also check min size because multer can't

    return next();
  }
}

export function MyFileUploadDecorator() {
  return UseBefore(CheckFileUploadMiddleware);
}

// Endpoint
@MulterOptions(getFileConfig(myEnum.A)) // with fileFilter
@MyFileUploadDecorator() // after MulterOptions
public uploadMyFile(
  @BodyParams('type') type: string, // keyof typeof MyEnum doesn't work
) {

In v.5 I suggest order: MyFileUploadDecorator, multer's fileFilter.

@MyFileUploadDecorator() // after MulterOptions
public uploadMyFile(
    @MultipartFile(
      getFileConfig(myEnum.A),
    )
    file: any, // PlatformMulterFile

Now it doesn't work. In v.6 I suggest order: multer's fileFilter, MyFileUploadDecorator.
In v.6 do I need to use Use instead of UseBefore? But if I want to use my decorator before MulterOptions? Do I need my own version of MulterOptions?
I use this order (MyFileUploadDecorator, multer's fileFilter) to for example don't upload files less than min size. Because in this case we need to add logic to delete them.

Thanks a lot.

[Romakita]

Hello,

For the problem with AJV:

@BodyParams('type') type: keyof typeof MyEnum ??

If it's an enum why don't you use type: MyEnum directly?

But to fix your problem, you an try this: @BodyParams('type', String) type: keyof typeof MyEnum

---

For the second point, to fix the issue #1189, I added the PlatformMulterMiddleware to default endpoint middlewares:

    handlers = handlers
      .concat(useCtxHandler(bindEndpointMiddleware(endpoint)))
      .concat(PlatformAcceptMimesMiddleware)
      .concat(hasFiles && PlatformMulterMiddleware) // here
      .concat(use) // Controller use-middlewares
      .concat(beforeMiddlewares) // Endpoint before-middlewares
      .concat(mldwrs) // Endpoint middlewares
      .concat(endpoint) // Endpoint metadata
      .concat(afterMiddlewares) // Endpoint after-middlewares
      .filter((item: any) => !!item);

https://github.com/TypedProject/tsed/blob/production/packages/common/src/platform/builder/PlatformControllerBuilder.ts#L96

You can see I the PlatformMulterMiddleware will be called always before all middlewares added to the endpoint. The multer options will consumed by PlatformMulterMiddleware.

Use or UseBefore won't change the problem. And I can't fix your issue and a the same time the #1189 issue.

See you
Romain

[Romakita]

@silveoj I created a PR which add unit test based on you example here: https://github.com/TypedProject/tsed/pull/1209/files

It works. So I don't understand the problem :)

See you
Romain

[silveoj]

I use keyof typeof MyEnum because frontend sends me not number, but string. This string is key from MyEnum.
So keyof typeof MyEnum validated only keys from MyEnum and didn't pass for example TITLE1 (see below) but pass TITLE

/** numeric */
enum MyEnum {
    TITLE,
    AGE,
}

type myNumType = MyEnum;
type myKeyType = keyof typeof MyEnum;

const numVal: myNumType = MyEnum.TITLE;
const keyVal: myKeyType = 'TITLE';

console.log(1, numVal); // 0 | 1
console.log(3, keyVal); // 'TITLE' | 'AGE'

Does it make sense for me to wrap MyFileUploadMiddleware and MulterOptions (PlatformMulterMiddleware )?
I suggest I can redefine PlatformMulterMiddleware? it not dangerous path?

Don't close this ticket yet. I will test new version of @tsed and check for file upload after 6.19.2

[Romakita]

Hoo ok. So I’ll fix the ajv to avoid your issue.
sure, I let the PR ;)

override middleware us possible but I don’t guarantee the breaking change on this middleware. But maybe I can add a hook event to allow any developper to handle the file. But the fileFilter doing the job in my opinion. His only problem, it’s not an injectable service, but it’s possible to get the req.$ctx.injector and get a service manually.

[silveoj]

Thanks. I thought about hook. I will try to moved my custom validation inside MulterOptions.

Maybe it's not issue with files.
But what about @BodyParams? They are always undefined for multipart/form-data (only for multipart/form-data, for application/json is fine). Even in 6.22.3. I can debug it if you know the best place in commit 6.19.2..6.20.0 say me)

 // application/form-data
  @Post('/upload-my-file')
  // @MulterOptions(
  //   ... commented
  // )
  public async uploadMyFile(
    @BodyParams('myDescription') myDescription: string, // undefined
    @BodyParams('myId') myId: string, // undefined
    // @MultipartFile('file') file: PlatformMulterFile, // commented
  ): Promise<boolean> {
    // ... all body params are undefined for form-data

@BodyParams('myDescription') @Any() myDescription: any, also doesn't work. It does not depend on acceptMimes array in Configuration.

[Romakita]

But what about @BodyParams? They are always undefined for multipart/form-data (only for multipart/form-data, for application/json is fine). Even in 6.22.3. I can debug it if you know the best place in commit 6.19.2..6.20.0 say me)

It depend on the bodyParser configuration. BodyParams decorator doesn't nothing, it just pick the req.body:
https://github.com/TypedProject/tsed/blob/production/packages/common/src/platform/services/PlatformHandler.ts#L123

Then the pipe parse the expression given to the decorator:
https://github.com/TypedProject/tsed/blob/production/packages/common/src/mvc/pipes/ParseExpressionPipe.ts

The better way to check if req.body is correctly fill is to add a pure function middleware on your endpoint:

  @Post('/upload-my-file')
  @UseBefore((req, res, next) => {
    console.log(req.body)
    next()
  })
  public async uploadMyFile(
    @BodyParams('myDescription') myDescription: string, // undefined
    @BodyParams('myId') myId: string
  ): Promise<boolean> {

  }
}

But again, the multipart works fine, this unit test confirm it:
https://github.com/TypedProject/tsed/blob/fix-1198-single-file/packages/platform-test-utils/src/tests/testMulter.ts#L111

See you
Romain

[silveoj]

I don't know but @MulterOptions is never called starting from 6.20.0.
In 6.19.2 it works fine except endpoint.get which will fixed in later release https://github.com/TypedProject/tsed/issues/1196.

// application/form-data @Post('/upload-my-file') @MulterOptions({ limits: { fileSize: myFileSizeConstant }, storage: multer.memoryStorage(), // or diskStorage fileFilter: (_req, _fileItem, callback) => { console.log('It never is called in 6.20.0 - 6.22.5 versions'); callback(null, true); }, }) public async uploadMyFile( @BodyParams('myDescription') myDescription: string, // undefined @BodyParams('myId') myId: string, // undefined @MultipartFile('file') file: PlatformMulterFile, // if commented my BodyParams can be undefined, can be not. ): Promise<boolean> { return true; }

I have 3 similar second projects. 2 have this issue with files. I didn't update third.

[Romakita]

Hello @silveoj
Ok if you have this issue on two projects, it only means, you have a similar configuration issue that I don't have!

I added new commit on the PR to demonstrate that the fileFilter function is indeed called:
https://github.com/TypedProject/tsed/pull/1209/commits/e7b8f6ce6499e8e2e8fbf91f33c42dc6ad8dfa9a

We're wasting time, give me a repository with the reproducible case and we can find the bug (we should have started there) ^^.

There are several possible reasons for the code not to perform as expected. I recently discovered that passport interferes with multer :).

I won't be surprised to find a reason related to your project configuration;)

See you
Romain

[daveruiz]

Hi all,

I'm reproducing this same issue in the version 6.22.2 (also tried the latest one, 6.23.0) and I'm able to consistently switch it on and off. I have a custom middleware that has some async tasks inside. If I bypass the async stuff making the middleware call next() directly, it works. If I call next() asynchronously then the file is missing in the controller.

This is my Server.ts file:

  public $beforeRoutesInit(): void | Promise<any> {
    this.app
      .use(cors(this.settings.cors))
      .use(cookieParser())
      .use(compress({}))
      .use(bodyParser.json({ type: 'application/json' }))
      .use(bodyParser.urlencoded({ extended: true }))
      .use(Fingerprint())
      .use(TimestampMiddleware)
      .use(CookieSessionMiddleware);

TimestampMiddleware and CookieSessionMiddleware are the only custom middlewares. CookieSessionMiddleware performs async tasks against DB, so it needs to be asynchronous.

The middleware looks like this (simplified to the minimum to reproduce the problem):

export class CookieSessionMiddleware implements IMiddleware {
 use(@Next() next: ExpressRequest) {
    setTimeout(next, 100);
  }
}

If I do return directly it works.

export class CookieSessionMiddleware implements IMiddleware {
 use(@Next() next: Next) {
    next();
  }
}

The controller has nothing special. Just something like this reproduces it:

  @Put('/upload') 
  async uploadTestPut(
    @Request() req: Request,
    @MultipartFile('file') file: Express.Multer.File,
  ) {
    return file;
  }

I've also tried the workaround above, of referencing PlatformMulterMiddleware directly but I'm getting a Cannot read property 'get' of undefined error in the import (similar to this other issue https://github.com/TypedProject/tsed/issues/1196).

[Romakita]

@daveruiz Don't add PlatformMulterMiddleware manually please. This is not the normal usage.

Again, give me a repository with reproducible example before adding more messages... I already covered this scenario with integration test, and all works fine!

See you
Romain

[Romakita]

@daveruiz Sorry. Your feedback give me maybe an idea of the problem, so thank you ;). I'll continue my investigation.

[daveruiz]

@Romakita Ok! Anyway, I've got a basic project that reproduces the issue. https://github.com/daveruiz/tsed-multer-issue

Thanks for your work here!

[Romakita]

@daveruiz Thanks for the repository ;). Thanks to your analysis I found the problem. It's related to the support of rawBody added recently. I'll fix that immediatly! thanks again for your help!

[Romakita]

Release is on going. I let you reopen the issue if the problem still occurs :)

[Romakita]

:tada: This issue has been resolved in version 6.23.1 :tada:

The release is available on:

• v6.23.1
• GitHub release

Your semantic-release bot :package::rocket:

[daveruiz]

I confirm it's working for me now, the sample project from above and the other project i'm working on.
Thanks so much for the quick fix!

[Romakita]

Great ;)