Ktor: MultiPartData.readAllParts() throws "java.io.IOException: Broken delimiter occurred" with empty multipart list

[SebastianAigner]

Problem description: When receiving an empty multipart-form request on a POST route, the request handler throws an IOException, preventing graceful handling or the display of an error message. Instead, a 500 error message is returned.

Minimum working example:

import io.ktor.application.*
import io.ktor.content.readAllParts
import io.ktor.response.*
import io.ktor.request.*
import io.ktor.routing.post
import io.ktor.routing.routing

fun main(args: Array<String>): Unit = io.ktor.server.netty.DevelopmentEngine.main(args)

fun Application.module() {
    routing {
        post("/add") {
            if(call.request.isMultipart()) {
                val multipart = call.receiveMultipart()
                val formItems = multipart.readAllParts() // <-- this breaks when sent an empty request
                call.respondText("recvd: $multipart")
            }
            else {
                call.respond("no multipart, fix request!")
            }
        }
    }
}

Send a request with an empty multipart body to see the issue:

curl -X POST \
  http://localhost:8080/add \
  -H 'Cache-Control: no-cache' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW'

Error Message:

22:57:11.866 [nettyCallPool-4-4] ERROR Application - Unhandled: POST - /add
java.io.IOException: Broken delimiter occurred
    at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.doResume(Delimited.kt:57)
    at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.invoke(Delimited.kt)
    at kotlinx.coroutines.experimental.io.DelimitedKt$skipDelimiterSuspend$2.invoke(Delimited.kt)
    at kotlinx.coroutines.experimental.io.ByteBufferChannel.lookAheadSuspend(ByteBufferChannel.kt:1746)
    at kotlinx.coroutines.experimental.io.DelimitedKt.skipDelimiterSuspend(Delimited.kt:55)
    at kotlinx.coroutines.experimental.io.DelimitedKt.skipDelimiter(Delimited.kt:50)
    at io.ktor.http.cio.MultipartKt.boundary(Multipart.kt:89)
    at io.ktor.http.cio.MultipartKt$parseMultipart$1.doResume(Multipart.kt:158)
    at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
    at kotlinx.coroutines.experimental.DispatchedKt.resumeCancellable(Dispatched.kt:209)
    at kotlinx.coroutines.experimental.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:35)
    at kotlinx.coroutines.experimental.CoroutineStart.invoke(CoroutineStart.kt:111)
    at kotlinx.coroutines.experimental.AbstractCoroutine.start(AbstractCoroutine.kt:165)
    at kotlinx.coroutines.experimental.channels.ProduceKt.produce(Produce.kt:95)
    at kotlinx.coroutines.experimental.channels.ProduceKt.produce$default(Produce.kt:88)
    at io.ktor.http.cio.MultipartKt.parseMultipart(Multipart.kt:145)
    at io.ktor.http.cio.MultipartKt.parseMultipart(Multipart.kt:138)
    at io.ktor.http.cio.CIOMultipartDataBase.<init>(CIOMultipartData.kt:33)
    at io.ktor.http.cio.CIOMultipartDataBase.<init>(CIOMultipartData.kt:31)
    at io.ktor.server.engine.DefaultTransformKt.multiPartData(DefaultTransform.kt:70)
    at io.ktor.server.engine.DefaultTransformKt.access$multiPartData(DefaultTransform.kt:1)
    at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.doResume(DefaultTransform.kt:33)
    at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.invoke(DefaultTransform.kt)
    at io.ktor.server.engine.DefaultTransformKt$installDefaultTransformations$2.invoke(DefaultTransform.kt)
    at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
    at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
    at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:64)
    at io.sebi.ApplicationKt$module$1$1.doResume(application.kt:31)
    at io.sebi.ApplicationKt$module$1$1.invoke(application.kt)
    at io.sebi.ApplicationKt$module$1$1.invoke(application.kt)
    at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
    at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
    at io.ktor.routing.Routing.executeResult(Routing.kt:100)
    at io.ktor.routing.Routing.interceptor(Routing.kt:25)
    at io.ktor.routing.Routing$Feature$install$1.doResume(Routing.kt:66)
    at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt)
    at io.ktor.routing.Routing$Feature$install$1.invoke(Routing.kt:51)
    at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
    at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
    at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.doResume(DefaultEnginePipeline.kt:66)
    at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
    at io.ktor.server.engine.DefaultEnginePipelineKt$defaultEnginePipeline$2.invoke(DefaultEnginePipeline.kt)
    at io.ktor.pipeline.PipelineContext.proceed(PipelineContext.kt:49)
    at io.ktor.pipeline.Pipeline.execute(Pipeline.kt:22)
    at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.doResume(NettyApplicationCallHandler.kt:31)
    at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt)
    at io.ktor.server.netty.NettyApplicationCallHandler$handleRequest$1.invoke(NettyApplicationCallHandler.kt:10)
    at kotlinx.coroutines.experimental.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:44)
    at kotlinx.coroutines.experimental.CoroutineStart.invoke(CoroutineStart.kt:113)
    at kotlinx.coroutines.experimental.AbstractCoroutine.start(AbstractCoroutine.kt:165)
    at kotlinx.coroutines.experimental.BuildersKt__Builders_commonKt.launch(Builders.common.kt:72)
    at kotlinx.coroutines.experimental.BuildersKt.launch(Unknown Source)
    at kotlinx.coroutines.experimental.BuildersKt__Builders_commonKt.launch$default(Builders.common.kt:64)
    at kotlinx.coroutines.experimental.BuildersKt.launch$default(Unknown Source)
    at io.ktor.server.netty.NettyApplicationCallHandler.handleRequest(NettyApplicationCallHandler.kt:22)
    at io.ktor.server.netty.NettyApplicationCallHandler.channelRead(NettyApplicationCallHandler.kt:16)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38)
    at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:844)

[sannysoft]

I'm having the same problem with ktor 0.9.4 with Netty

[sannysoft]

Some details on the issue:
The problem only happens for me when I first call receiveParameters & later call receiveMultipart
Here is a simple example:

val data = call.receiveParameters()
val multipart = call.receiveMultipart()
multipart.forEachPart { println(it.name) }

[hathub]

Some details on the issue:
The problem only happens for me when I first call receiveParameters & later call receiveMultipart
Here is a simple example:

val data = call.receiveParameters()
val multipart = call.receiveMultipart()
multipart.forEachPart { println(it.name) }

Same problem calling first "call.receiveParameters()"

[comm1x]

@SebastianAigner You send incorrect multipart request.

Curl handling boundary itself. And when you set manually -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' curl add one more additional boundary, and header Content-Type in request looks like

Content-Type: multipart/form-data; boundary=WebKitFormBoundary7MA4YWxkTrZu0gW; boundary=------------------------8b5ba40884633a0e

With double boundary. It is invalid multipart. You can see this using -v flag in curl.
Currently in 1.0.0-rc1 ktor throws java.io.IOException: Multipart preamble/prologue limit of 8192 bytes exceeded for this case.

[AlexanderMatveev]

Official docs example (https://ktor.io/servers/uploads.html) throws same exception.

val multipart = call.receiveMultipart()
multipart.forEachPart { part ->
    when (part) {
        is PartData.FormItem -> {
            if (part.name == "title") {
                title = part.value
            }
        }
        is PartData.FileItem -> {
            val ext = File(part.originalFileName).extension
            val file = File(uploadDir, "upload-${System.currentTimeMillis()}-${session.userId.hashCode()}-${title.hashCode()}.$ext")
            part.streamProvider().use { input -> file.outputStream().buffered().use { output -> input.copyToSuspend(output) } }
            videoFile = file
        }
    }

    part.dispose()
}


suspend fun InputStream.copyToSuspend(
    out: OutputStream,
    bufferSize: Int = DEFAULT_BUFFER_SIZE,
    yieldSize: Int = 4 * 1024 * 1024,
    dispatcher: CoroutineDispatcher = Dispatchers.IO
): Long {
    return withContext(dispatcher) {
        val buffer = ByteArray(bufferSize)
        var bytesCopied = 0L
        var bytesAfterYield = 0L
        while (true) {
            val bytes = read(buffer).takeIf { it >= 0 } ?: break
            out.write(buffer, 0, bytes)
            if (bytesAfterYield >= yieldSize) {
                yield()
                bytesAfterYield %= yieldSize
            }
            bytesCopied += bytes
            bytesAfterYield += bytes
        }
        return@withContext bytesCopied
    }
}

HTML form:

 <form action="/upload" method="post" enctype="multipart/form-data">
                <input type="file" name="file"/>
                <button type="submit">Upload</button>
</form>

[AlexanderMatveev]

Looks like Ktor is not very actively maintained framework :(

[oleg-larshin]

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.