Ktor: Multipart not reading all/any parts

[monarezio]

Hi, I am trying to upload an image/file...

I ran into a problem that the multiplart.readAllParts() returns everytime empty list, even when I send a file! I migrated from a version 0.3.2 (I think) to 0.9.1... I have read this issue: https://github.com/ktorio/ktor/issues/139 but it seems as no one has answered the question.

I have even tried creating a form with the correct enctype and uploading the file through the form, without any success. I also tried uploading the file through $.ajax() method in jqeury, also without any success.

My code:

if(call.request.isMultipart()) {
                val multipart = call.receiveMultipart()
                println("FILES:" + multipart.readPart())
                multipart.forEachPart { part ->
                    when (part) {
                        is PartData.FileItem -> {
                            files += part
                        }
                    }
                }
            }

I was reading this documentation: http://ktor.io/servers/uploads.html

btw, thank you very much for developing ktor! I am developing my graduation project in it!

[betterclever]

Can confirm this issue.

[soywiz]

Have you resolved this issue already? If it is not the case, please give more information to reproduce it: which version of Ktor are you using, which engine are you using? Netty, Jetty, CIO...?

Also have you tried the youkube example? Does multipart file upload work for you in that sample?
https://github.com/ktorio/ktor/tree/master/ktor-samples/ktor-samples-youkube

Also in your code I see a problem, but I suppose it was just because of the tests: if you read one part (println("FILES:" + multipart.readPart())), that part won't be available later on the forEachPart code. It is like a stream, you read it once and later it returns null everytime. It is like that to avoid having to keep everything in memory or in files, so you have to process it once and in order. The forEachPart definition looks like this:

suspend fun MultiPartData.forEachPart(partHandler: suspend (PartData) -> Unit) {
    while (true) {
        val part = readPart() ?: break
        partHandler(part)
    }
}

[betterclever]

I am using Ktor version: 0.9.2-alpha-2 with Netty engine.
My code looks like:

route("/files") {
                    post("/new") {
                        val multipart = call.receiveMultipart()
                        var title = ""
                        var videoFile: File? = null

                        multipart.forEachPart { part ->
                            when (part) {
                                is PartData.FormItem -> if (part.name == "title") {
                                    title = part.value
                                }
                                is PartData.FileItem -> {
                                    val ext = File(part.originalFileName).extension
                                    val file = File(uploadDir, "upload-${System.currentTimeMillis()}-${"AVX".hashCode()}-${title.hashCode()}.$ext")
                                    part.streamProvider().use { its -> file.outputStream().buffered().use { its.copyTo(it) } }
                                    videoFile = file
                                }
                            }

                            part.dispose()
                        }

                        call.respond("done")
                    }
                }

I tried putting some debugging breakpoints. The call doesn't seem to enter the forEachPart and server is stuck with no logs whatsoever. I am sending request with Postman and can confirm same multipart request works with my nodejs express server.

I tried running YouKube example. While running example, the browser seems to be stuck here pointing towards this issue only.

[monarezio]

I am using Ktor version 0.9.1 using Netty engine
In the previous version of ktor, before I upgraded, the upload worked fine... I will try to clone the repo with the example later today, when I get more time...

The "problem" with my code is because I already tried debuging it somehow (testing as you mentioned already) 😃

[monarezio]

I reproduced the same bug as @betterclever

Ktor: 0.9.1
Engine: Netty

I guess its because of the readPart() It returns everytime null, I tried looking deeper into the code, but, since I did not design Ktor, it is very hard for me to understand the code 😕.

[betterclever]

@soywiz Please check this issue. I can confirm this issue from version 0.9.1 above of Ktor. I tried on Linux, macOS and windows as well. The YouKube sample itself is not working. :disappointed:

[soywiz]

Didn't test the sample myself.

I have just tested it, and I can confirm it doesn't work.

We are going to check it.

[cy6erGn0m]

Youkube sample is fixed now in master. @monarezio @betterclever Could one please verify that it is fixed in your application?

[betterclever]

Any way to use Ktor dependency from Git source?

[cy6erGn0m]

You can try jitpack ( https://jitpack.io/#ktorio/ktor )

[monarezio]

@cy6erGn0m I used jitpack.io to download the latest git build of ktor:netty-server through gradle and it seems that I get stuck on the method: call.receiveMultipart().

My code:

if(call.request.isMultipart()) {
                println("IS MULTIPART")
                val multipart = call.receiveMultipart()
                println("MULTIPART: ${multipart}")
                multipart.forEachPart { part ->
                    println("PART: ${part}")
                    when (part) {
                        is PartData.FileItem -> {
                            files += part
                        }
                    }
                    part.dispose()
                }
            }

The code only outputs 'IS MULTIPART' but after that nothing...

Are you encoutering this too @betterclever ?

[cy6erGn0m]

Then I need an example to reproduce it

[soywiz]

I have tried the youkube sample and now it is working. In addition to jitpack you can build it from source. I have added an entry for this in the FAQ: https://ktor.io/quickstart/faq.html#bleeding-edge

The main idea is to download the ktor's GIT repo, verify that the gradle.properties has this as the first line: version=0.9.2-SNAPSHOT.
Now you can install it to your maven local repository with: ./gradlew install

Then you can download ktor-samples repository and change the gradle.properties so the ktor_version points to 0.9.2-SNAPSHOT: ktor_version = 0.9.2-SNAPSHOT.
Then you have to tell gradle to search for artifacts in maven local. You can add this to the root build.gradle in the ktor-samples repo:

allprojects {
    repositories {
        mavenLocal()
    }
}

And since in master now authentication works differently, you have to update the app/youkube/src/Login.kt:

This should work:

fun Route.login(users: UserHashedTableAuth) {
    application.install(Authentication) {
        form("login") {
            userParamName = Login::userName.name
            passwordParamName = Login::password.name
            challenge = FormAuthChallenge.Redirect { call, c -> call.url(Login(c?.name ?: "")) }
            validate {
                users.authenticate(it)
            }
        }
    }
    location<Login> {
        authenticate("login") {
            post {
                val principal = call.principal<UserIdPrincipal>()
                call.sessions.set(YouKubeSession(principal!!.name))
                call.respondRedirect(Index())

And then you can run youkube:

./gradlew youkube:run

I have tested on macos + chrome and it works flawlessly.

Please verify that you are using the right version checking the Server header sent when installing the DefaultHeaders feature: install(DefaultHeaders).

If youkube works for you, but still your own code doesn't work, please provide a way to reproduce it locally.

Thanks for reporting!

[monarezio]

@soywiz @cy6erGn0m yep it works, I was just dumb and called call.receive() twice in one call

Thank you for solving this issue!