Gin: How to retrieve multiple files continuously

[HuanjunKong]

After checking #562 , I post 10 images (500KB) to gin, code :

    func extractImgs(c *gin.Context) {
        prefix := "prefix"
        for ix := 0; ix < 10; ix++ {
            file, _, err := c.Request.FormFile(prefix + strconv.Itoa(ix))
            if err != nil {
                // do sth.
                return
            }
            // call face recognition algorithm
            face_rec_async(file)
        }
    }

c.Request.FormFile cost a lot time. It seems like that FormFile returns file after retrieved all 10 files.

Face recognition is time-consuming algorithm, this workflow should be better:

retrieve first file -> recognize face -> retrieve second file -> recognition -> ... ,

just like a link list.

My question is how to retrieve files like traverse linked list.

[Michael77]

I have found a way that works by turning the multipart form into a reader. Here is a code snippet without full error handling:

reader, _ := c.Request.MultipartReader()

for {
    if part, err := reader.NextPart(); err == io.EOF {
        break
    }

    // part implements io.Reader
}

[javierprovecho]

@HuanjunKong sorry for late reply, .FormFile() is a wrap from net/http. If the form hasn't been parsed yet, it process all files by creating for each one a File object. After looking through the code, here should not be any delay after calling for the first file.

@Michael77 is another solution but I see that you could not access directly a file by name.

[jeffkoncz]

I had to do something similar--transferring multiple file uploads to S3 in parallel. Adding to @Michael77 method if you need access to the original filename you can use the mime package.

disposition, params, err := mime.ParseMediaType(part.Header.Get("Content-Disposition"))
if err != nil {
  break
}
filename := params["filename"]

[svperfecta]

hey @jeffkoncz any chance you have a larger example of this? I'm a bit stuck, doing the exact same thing.

[appleboy]

@genexp You can see the example https://github.com/gin-gonic/gin/blob/develop/examples/upload-file/multiple/main.go

[jeffkoncz]

@genexp Here's some code that is specific to transferring multiple file uploads to S3. Hopefully it can get you unstuck :) :

func MultipartUploadHandler(c *gin.Context) {
    multipart, err := c.Request.MultipartReader()
    if err != nil {
        log.Fatalln("Failed to create MultipartReader", err)
    }

    for {
        mimePart, err := multipart.NextPart()
        if err == io.EOF {
            break
        }
        if err != nil {
            log.Printf("Error reading multipart section: %v", err)
            break
        }
        disposition, params, err := mime.ParseMediaType(mimePart.Header.Get("Content-Disposition"))
        if err != nil {
            log.Printf("Invalid Content-Disposition: %v", err)
            break
        }

        // S3 Upload Manager
        uploader := s3manager.NewUploader(session.New(&aws.Config{Region: aws.String("us-west-2")}))
        result, err := uploader.Upload(&s3manager.UploadInput{
            Body:   mimePart,
            Bucket: aws.String("yourbucket"),
            Key:    aws.String(params["filename"]),
            ContentType: aws.String(mimePart.Header.Get("Content-Type")),
            ACL: aws.String("public-read"),
        })
        if err != nil {
            log.Fatalln("Failed to upload to S3", err)
        }

        c.JSON(http.StatusOK, gin.H{
            "location": result.Location,
        })
    }
}

[svperfecta]

@jeffkoncz Yeah, this is awesome. Thanks so much!

[geoherna]

Awesome! Thanks!