Ktor: Support native windows HTTP client

[dtretyakov]

To create native windows applications could be useful to have native HTTP client which supports using Windows certificates store for SSL connections, proxy settings configuration on the system level, etc.

Microsoft recommends using WinHTTP for server-side applications and services. WinHTTP is used in .NET windows http client so it's a good candidate to use it on Windows. WinHTTP could work in sync and async mode, which is recommended to use and async calls could be wrapped by using Kotlin coroutines.

[dtretyakov]

The sample kotlin native application which provides WinHTTP-based client is available here: https://github.com/dtretyakov/kotlin-winhttp

Currently it has the following problems:

• To use suspend functions in kotlin native coroutines should be supported multi-threading working mode, since WinHTTP uses pool of worker threads and completion callback could be executed on any of them: (https://github.com/Kotlin/kotlinx.coroutines/issues/462)
• WinHTTP headers are not included as Windows platform library in kotlin native, so it requires having cintorop library. I'm not sure where is a better place to have winhttp interop in the windows platform or in this repository.
• In WinAPI most of calls returns string as UTF16 and I it looks like kotlin native library is missing helper functions to create Kotlin strings from ShortArrays and it requires constructing string like that:

val buffer = allocArray<ShortVar>(size)
// Call to fill in the buffer
String(CharArray(size) {
    buffer[it].toChar()
})

[cy6erGn0m]

Kotlinx I/O supports string constructor String(ByteArray, Charset), perhaps I can support construction from any CPointer as well

[cy6erGn0m]

I believe it's ok to have winhttp cinterop in the client

[cy6erGn0m]

To get it work with coroutines a set of hacks need to be applied: currently coroutines only work in the main thread so you need to send signals from a callback thread. The other issue is that there is no main loop implemented on Windows so it's not clear who can listen to such signals.

[cy6erGn0m]

@dtretyakov Does winhttp support streaming for request and response body?

[dtretyakov]

@cy6erGn0m, WinHTTP async read/write operations are based on request -> callback response style of invocation: https://msdn.microsoft.com/en-us/library/windows/desktop/aa383138%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396#art_winhttpheaders

So it looks like e.g. the response body could be consumed as a stream while invoking series of:

WinHttpQueryDataAvailable(...)
// -> WINHTTP_CALLBACK_STATUS_DATA_AVAILABLE callback
// allocate buffer
WinHttpReadData(...)
// -> WINHTTP_CALLBACK_STATUS_READ_COMPLETE callback
// read from the buffer

until WINHTTP_CALLBACK_STATUS_DATA_AVAILABLE returns 0 for statusInfo

[cy6erGn0m]

Note that we are going to introduce curl client, see #777

[e5l]

Merged in master

[msink]

Why was closed?
On Windows libcurl from msys2-mingw is not well integrated and hard to redistribute - good enought for prototyping, but not so good for production.
So client implementation based on winhttp or wininet still required.

[dtretyakov]

@cy6erGn0m, @e5l I've added preliminary PR to support WinHTTP-based client: https://github.com/ktorio/ktor/pull/957

[Dominaezzz]

Is the WinHTTP client going to be released soon or has it been indefinitely put aside?

[PatrickLemke]

Just to clarify: There is currently no ktor client available for windows, correct?

[e5l]

CURL client engine supports Windows.

[PatrickLemke]

That's interesting, since if I include both required imports:

mingwX64().compilations["main"].defaultSourceSet { dependencies { implementation("io.ktor:ktor-client-core:$ktorVersion") implementation("io.ktor:ktor-client-curl:$ktorVersion") } }

I can still not import io.ktor.* Is there anything else I need to take into consideration?

[msink]

CURL client engine supports Windows

Does not support https:, only http: works.

[PatrickLemke]

@msink https works for me

[msink]

@PatrickLemke - it works in debug mode, but I failed to make distribution package.
Do you have any example how to distibute it?

[PatrickLemke]

@msink I haven't tried that yet. Are you referring to the difficulty of installing msys on the client system? I was thinking to use an installer like NSIS to install it manually.

[msink]

If I remember correctly - it works only if .exe was copied to msys2 bin/ directory, and runned from here. Something like as Git for Windows is distributed.

[PatrickLemke]

Well I guess you could also copy all the libraries you need into your own folder. The only thing that should matter is that mysys has access to all the libraries it needs.

[msink]

Not exactly - it looks for ssl certificates in directory relative to .exe - ../ssl , so .exe should be in subdirectory, for example bin/

[oleg-larshin]

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.